Spell check can't tell that “form” was meant to be “from”,
since both are real words. This 2.1 MB model can.
Write with it in offlinepage.app → · Try the demo →
tinyfix reads the words on both sides of a correctly spelled word and picks the best fit from nearby real words. It runs on the page with ONNX Runtime Web, so nothing you type is sent anywhere.
What it does
| You type | It becomes |
|---|---|
| I will send it |
I will send it from home tomorrow. |
| There is a problem with the invoice. | |
| I am |
I am too tired to cook tonight. |
| I think |
I think you're right about that. |
| Please fill out the form and return it. | No change. “form” is right here. |
How well it works
94%of its fixes are right |
40%of wrong words fixed24% in the first version |
~3 msper check in Chrome |
Measured by typing test text into the same code offlinepage.app runs, on text the model never trained on and that nothing was tuned on. The 40% is the average of five test sets:
| Test set | Wrong words fixed | Fixes that were right |
|---|---|---|
| Classic mix-ups put into email, chat and encyclopedia text (their/there, to/too, its/it's…) | 69% | 99% |
| Hand-written sentences | 58% | 100% |
| Finger slips that make a real word | 29% | 87% |
| Real errors from children's writing | 28% | 91% |
| Real typos from GitHub commits | 18% | 96% |
On about 110,000 words of correct email, chat, encyclopedia, news, documentation and fiction, it wrongly changed about 1 word in 2,000.
The first two sets are built from the mix-up pairs the model was trained to watch for, so they flatter it. The last three are the fairer picture of everyday typing.
What it misses
- It still misses more wrong words than it catches, and about four in five real typos from GitHub commits.
- When both words make sense it stays quiet: “writing this form a plane” is left alone, because “writing this form” is something people say.
- Two-letter words are hit and miss. It fixes “I am to tired” but not every “to” that should be “too”.
- It can only judge the 20,000 words it knows, so a rarer word like “ware” typed for “were” is left alone.
- It needs the words that come after, so these fixes land a couple of words later.
- English only.
How it works
Autocorrect in offlinepage.app is two parts, and this model is the second one.
- Misspellings (“recieve”) are caught without a model: an 80,000-word list, a search for words within one or two edits, and a cost for each edit based on how likely that slip is on a keyboard.
- Real words in the wrong place (“form” for “from”) are what tinyfix is for. For each finished word, the app gathers up to eight nearby real words: known mix-ups first (their/there/they're), then words one edit away. tinyfix scores the typed word and each candidate against the 16 words before and up to 4 words after.
- The swap is only made when the gap is clear. The best candidate has to beat the typed word by a set margin, and a less likely slip needs a bigger margin. That margin is what keeps wrong changes to about 1 word in 2,000.
The app waits for two more words, or the end of the sentence, before it checks a word. That is why a fix lands a moment after the typo.
How it was trained
The task. Take a sentence of ordinary text, pick a word, and half the time swap it for a nearby real word. The model sees the words around the gap and a short list of candidates, and learns to score the word that was really there highest. The other half of the time nothing is swapped, which teaches it to leave correct words alone.
The data. About 120 million words of plain text, with no labels: 70 million from FineWeb-Edu articles and 50 million from SmolTalk conversations, plus OpenAssistant messages.
The recipe that mattered. One training example in four is centered on a word from 160 groups of words people mix up, and half of those show the word used correctly. This one change did most of the work: it took the fix rate on classic mix-ups from 25% to 69%.
The model. Two small GRUs, one reading left to right up to the word and one reading right to left back to it, joined and compared against each candidate's embedding.
| Setting | Value |
|---|---|
| Parameters | 1.9 million |
| Vocabulary | 20,000 words, lowercased; names and numbers each become one token |
| Embedding / hidden size | 64 / 256 |
| Context | 16 words before, 4 after |
| Training | 120,000 steps, batch 4,096, learning rate 0.002 |
| Time | about 1 hour 45 minutes on one A10G, with Hugging Face Jobs |
| Download | 2.1 MB after 8-bit quantization (the GRU is built from plain matrix multiplies so ONNX Runtime can quantize it) |
How it was measured. A bench that types frozen test text through the app's real code, one keystroke at a time, and counts what got fixed and what got wrongly changed. Settings were tuned on a separate development split. Two runs of the same recipe differ by 2 to 3 points, so smaller differences were treated as noise.
What was tried and didn't help
Each of these scored the same as the recipe above, within the noise between two identical runs:
- A wider embedding (96 instead of 64), which also made the file bigger
- Distilling from BERT
- More context after the word
- Training on real typo pairs from GitHub commits
- Simulated keyboard slips as training swaps
- Training for longer, and averaging weights across the run
Most of the remaining misses come from two limits outside training. The right word often isn't among the candidates offered, and a typo outside the 20,000-word vocabulary can't be judged at all.
Using the model
The easiest way is the tinyfix package, which loads this model and returns the fixes for a text:
import { loadTinyfix } from "tinyfix";
const tinyfix = await loadTinyfix();
await tinyfix.fix("I got a letter form the bank."); // "I got a letter from the bank."
To call the model yourself: it scores candidate words for one position in a sentence. Higher scores fit better, and comparing each candidate to the typed word tells you whether a swap is worth making.
Inputs and outputs
| Name | Shape | What it holds |
|---|---|---|
left (input) |
[batch, 16] int64 |
The words before, left-padded with 0, starting with <s> when there are fewer than 16 |
right (input) |
[batch, 4] int64 |
The words after, nearest word last, left-padded with 0 |
candidates (input) |
[batch, 9] int64 |
The typed word first, then up to 8 nearby words, padded with 0 at the end |
scores (output) |
[batch, 9] float |
One score per candidate |
IDs are line numbers in wrong-word-vocab.txt, starting from 0. Words are lowercased, and capitalized words in the middle of a sentence become <name>.
Files
| File | What it is |
|---|---|
wrong-word.onnx |
The quantized model |
wrong-word-vocab.txt |
The 20,000 words it knows, one per line |
words.txt |
The 80,000-word spelling list, with how common each word is |
wrong-word-confusions.txt |
About 235 words people mix up, like “their” and “there”, each with the words they are mistaken for |
train_realword.py |
The training script |
The model only scores words. The rules that decide when to make a swap are in the tinyfix package.
tinyfix is one of two tiny models behind offlinepage.app, a writing app that works in airplane mode.
Its partner, tinynext, suggests the next few words.
Both are made by paperweight, purpose-built small models.