The American Sign Language alphabet, recognised in this tab. Nothing is uploaded — there is no server to upload it to.
MODEL 52,000 params · 24 letters TRACKER MediaPipe 0.10.21 INFERENCE on-device FPS —
Evidence
Sign-language accuracy is usually quoted from a random train/test split. Public ASL datasets are frames pulled from a handful of continuous recording sessions, so neighbouring frames are near-duplicates of one another. Split them randomly and almost every test image has a near-twin in training: the model scores beautifully, then falls apart on a stranger's hand. Both numbers are here, because the contrast is the honest answer.
on a person the model has never seen
One dot per held-out person. The spread between them is the honest measure of how much a single reported figure can be trusted.
Pooled across all five held-out folds, worst first. Every letter scores between 70% and 99%, so this plots what is left over rather than 24 near-identical full bars.
The readout shows a percentage beside every letter. A softmax is not a probability just because it sums to one, so this is measured the same way everything else here is — across held-out signers. Each row is a band of claimed confidence against how often predictions in that band were actually right.
A letter is only committed while the model is above a confidence floor. Raising it trades frames away for accuracy; this is the trade at each setting.
Rows are the true letter, columns what the model said. The diagonal is drawn in neutral grey because it is not the interesting part; errors carry the amber scale, capped so a 5% mistake is still visible. Hover any cell.
Share of the true letter that went to the wrong one.
R and U are crossed fingers against two raised ones. P and Q are close to one hand shape at two orientations — and the rotation step that lets the model survive a new person is what erases the difference.
Every figure on this page is read at load time from eval/results.json, the output of
ml/evaluate.py, so the page cannot claim an accuracy the repository did not
measure. The full protocol and limitations are in
docs/EVALUATION.md,
which is generated from the same file and doubles as the table view of these charts.
Reference
Every hand here is a real pose from the training data — the sample closest to the average of its letter — redrawn from its landmarks rather than illustrated. The bar under each cell is that letter's measured accuracy. Pick one to see it larger.
Method
It returns 21 landmarks — knuckles, joints, fingertips — as 3D coordinates. This is the expensive part and it runs on your GPU. It was trained on a corpus far larger and more varied than this project could assemble, which is why it is borrowed rather than rebuilt.
The wrist becomes the origin, the hand is rotated so it points up, and it is scaled so its farthest landmark sits at distance 1. Left hands are mirrored onto right. What survives is shape alone — not where the hand was, how big, how tilted, or which hand.
63 numbers into two hidden layers into 24 letters. It never sees a pixel, which is exactly why it cannot learn one person's skin, room or lighting — and why it trains on a laptop CPU in minutes rather than needing a GPU.
Rotating orientation away is what lets the model survive a stranger holding their wrist at a different angle. It is also why P and Q get confused: those two signs are close to one hand shape at two orientations, and step two erases the difference. The generalisation and that specific failure are the same decision seen from both sides. J and Z are absent entirely — both are drawn in the air, and this model sees one frame at a time.