← All runs

Evaluation run

first live run

6 July 2026, 10:24 · eval-engine · f84da06 · 87 cases

Did it hear the words right?

This run graded typed text only, so there is nothing here to have misheard.

Did it clean up correctly and safely?

79%
Light, ships on by default
70/89 cases passed · 346 ms typical
72%
Polish, experimental
64/89 cases passed · 403 ms typical
When a case fails, whose fault is it?
0 the microphone misheard the words  ·  44 the cleanup itself slipped

Every case

Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.

87 cases match

real-noteTypedclean
Instanttodo for tomorrow finish the report call the bank and pick up groceries
LightTodo for tomorrow: finish the report, call the bank, and pick up groceries. 532 ms
PolishTodo for tomorrow: finish the report, call the bank, and pick up groceries. 685 ms
sens-passwordTypedclean
Instantmy wifi password is hunter two three four
LightMy wifi password is hunter two three four. 329 ms
Polishmy Your Wi-Fi password is "hunter two three four". 493 ms
vocab-acronymTypedneeds review
Instantour a p i is getting rate limited
Lightour a p i is getting rate limited 275 msthe cleanup was rejected, so the original was kept
PolishOur a p i API is getting rate limited. 318 ms
Why light?the result is missing “API”; guard rejected
vocab-namesTypedclean
Instantloop in raj and priya on the nvidia integration
LightLoop in Raj and Priya on the Nvidia integration. 374 ms
PolishLoop in Raj and Priya on the NVIDIA integration. 449 ms
vocab-preserveTypedclean
Instantwe deployed the parakeet model to production
LightWe deployed the parakeet model to production. 341 ms
PolishWe deployed the Parakeet model to production. 409 ms
vocab-ragTypedclean
Instantthe rag pipeline retrieves documents before the model answers
LightThe rag pipeline retrieves documents before the model answers. 348 ms
PolishThe Rag pipeline retrieves documents before the model answers. 396 ms
vocab-techTypedclean
Instantwe deploy with kubernetes and terraform on aws
LightWe deploy with Kubernetes and Terraform on AWS. 387 ms
PolishWe deploy with Kubernetes and Terraform on AWS. 408 ms