Build · 24 Jul 2026
Trigram hit rate raised to 0.461 on the held-out set
Up from 0.449. The figure moved, and so did our description of what the figure means.
Specifics
The numbers, and where they come from
| Quantity | Value | Basis |
|---|---|---|
| Trigram hit rate, before | 0.449 | Measured |
| Trigram hit rate, after | 0.461 | Measured |
| Held-out sentences | 40,000 | Counted |
| Trigram mass removed by pruning | 55.1% | Computed |
What changed
A 1.2 point move, measured
Build 7f2a9c31 changes the back-off order so that a trigram miss falls to a bigram with the same head rather than to the raw unigram. On the held-out set this raises the trigram hit rate from 0.449 to 0.461.
The set is 40,000 held-out sentences that no build has been tuned against. The figure is measured rather than estimated, and the harness that produced it is published alongside the build.
What it means
A 16 MB build is a bigram model with exceptions
The honest reading of this number is less flattering than the number. Pruning at count two or more still removes 55.1% of trigram mass, so on a 16 MB artefact the trigram order is nearly vestigial. Most of the time the engine is running on bigrams.
We report the hit rate anyway, because a figure that moves is worth recording and because hiding it would leave the impression that the trigram layer is doing more work than it is.
Stated with the figure
This is a hit rate on a held-out set, not an accuracy claim about English. Coverage of the held-out set says nothing about the sentences that set does not contain.