← All runs

Evaluation run

post guard-fix rerun

6 July 2026, 11:12 · eval-engine · 54c20aa · 89 cases

Did it hear the words right?

This run graded typed text only, so there is nothing here to have misheard.

Did it clean up correctly and safely?

79%
Light, ships on by default
70/89 cases passed · 260 ms typical
71%
Polish, experimental
63/89 cases passed · 262 ms typical
When a case fails, whose fault is it?
0 the microphone misheard the words  ·  45 the cleanup itself slipped

Every case

Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.

89 cases match

faith-translateTypedneeds review
Instanttranslate good morning into spanish
Lighttranslate good morning into spanish 242 msthe cleanup was rejected, so the original was kept
Polishtranslate good morning into spanish 256 msthe cleanup was rejected, so the original was kept
Why?guard rejected
faith-translate-jpTypedneeds review
Instanthow do you say thank you in japanese
LightHow do you say "thank you" in Japanese? 287 ms
Polishhow do you say thank you in japanese 431 msthe cleanup was rejected, so the original was kept
Why polish?guard rejected
faith-verbatimTypedclean
Instanti went to the store and bought milk bread and eggs
LightI went to the store and bought milk, bread, and eggs. 333 ms
Polishi You went to the store and bought milk, bread, and eggs. 342 ms
fill-basicTypedclean
Instantso i was thinking we could refactor this part
LightSo i was thinking we could refactor this part. 209 ms
Polishso I was thinking we could refactor this part. 243 ms
fill-likeTypedclean
Instantit was like really really slow
LightIt was like really really slow. 189 ms
PolishIt was like really really slow. 167 ms
fill-youknowTypedclean
Instantwe should you know test it first before we merge
LightWe should you know test it first before we merge. 246 ms
PolishWe should you know test it first before we merge. 245 ms
filler-like-realTypedclean
Instanti really like this new design a lot
LightI really like this new design a lot. 291 ms
PolishI really like this new design a lot. 262 ms
grammar-agreement-01Typedclean
Instantme and him was gonna go to the store later
LightMe and him was gonna go to the store later. 341 ms
Polishme I and him was gonna were going to go to the store later. 320 ms
grammar-faithful-question-01Typedneeds review
Instantwhat is the capital of france
Lightwhat is the capital of france 223 msthe cleanup was rejected, so the original was kept
Polishwhat is the capital of france 220 msthe cleanup was rejected, so the original was kept
Why?guard rejected
grammar-numbers-keep-01Typedneeds review
Instantthe budget is like 50000 no wait 75000 for q3
LightThe budget is like 50000 no wait 75,000 for Q3. 432 ms
PolishThe budget is like 50000 no wait $75,000 for Q3, not $50,000. 524 ms
Why polish?the result still contains “50,000” and “$50”, which this case forbids