← All runs

Evaluation run

ITN room-number fix

6 July 2026, 13:19 · eval-engine · 54c20aa · 89 cases

Did it hear the words right?

This run graded typed text only, so there is nothing here to have misheard.

Did it clean up correctly and safely?

83%
Light, ships on by default
74/89 cases passed · 255 ms typical
75%
Polish, experimental
67/89 cases passed · 259 ms typical
When a case fails, whose fault is it?
0 the microphone misheard the words  ·  37 the cleanup itself slipped

Every case

Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.

89 cases match

caps-properTypedclean
Instanti met sarah in new york on tuesday
LightI met Sarah in New York on Tuesday. 254 ms
PolishI met Sarah in New York on Tuesday. 255 ms
corr-money-noTypedneeds review
Instantthe plan is 25 no $40 a month
LightThe plan is 25 no $40 a month. 255 ms
PolishThe plan is $25 no $40 a month. 259 ms
Why polish?the result is missing “40”; the result still contains “25”, which this case forbids
corr-name-chainTypedneeds review
Instantcall john no jane no actually mike
LightCall John, no Jane, no actually Mike. 266 ms
PolishCall John, no not Jane. No, actually, Mike. 306 ms
Why?the result still contains “John” and “Jane”, which this case forbids
corr-num-chainTypedneeds review
Instantwe need 20 no 30 no 40 units
LightWe need 20, no 30, no 40 units. 418 ms
PolishWe need 20 no 30 no 40 units. 222 ms
Why light?the result still contains “20” and “30”, which this case forbids
Why polish?the result is missing “40”; the result still contains “30”, which this case forbids
corr-time-noTypedclean
Instantlet's meet at 3 no 4:30
LightLet's meet at 3 no 4:30. 271 ms
PolishLet's meet at 3 no 4:30. 276 ms
disf-dayTypedclean
Instantlet's do it on monday sorry i mean tuesday
LightLet's do it on monday sorry i mean Tuesday. 216 ms
PolishLet's do it on monday sorry i mean Tuesday. 209 ms
disf-false-startsTypedclean
Instantso the the main thing is the main point is we need to ship
LightSo the the main thing is the main point is we need to ship. 284 ms
PolishSo the the main thing is the main point is that we need to ship. 303 ms
disf-heavy-stutterTypedneeds review
Instanti i i think we we should just go go now
LightI i i think we we should just go go now. 215 ms
PolishI i i think we we should just go go now. 247 ms
Why light?the result is missing “I think we should just go now”
disf-repeatTypedclean
Instantthe the report is is ready to go
LightThe the report is is ready to go. 212 ms
PolishThe the report is is ready to go. 209 ms
disf-restartTypedclean
Instanti think we should i think we should just ship it
Lighti think We should i think we should just ship it. 202 ms
Polishi think We should i think we should just ship it. 199 ms