← All runs

Evaluation run

faithfulness scoring cleanup

6 July 2026, 13:36 · eval-engine · 059e772 · 89 cases

Did it hear the words right?

This run graded typed text only, so there is nothing here to have misheard.

Did it clean up correctly and safely?

94%
Light, ships on by default
84/89 cases passed · 254 ms typical
89%
Polish, experimental
79/89 cases passed · 266 ms typical
When a case fails, whose fault is it?
0 the microphone misheard the words  ·  15 the cleanup itself slipped

Every case

Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.

89 cases match

grammar-runon-01Typedclean
Instantso i went to the meeting and then i talked to sarah and then we decided to push the launch and also we need to tell the team
LightSo I went to the meeting and then I talked to Sarah and then we decided to push the launch, and also we need to tell the team. 684 ms
PolishSo I went to the meeting and then i talked to Sarah. and then We decided to push the launch and also we need to tell the team. 637 ms
grammar-tense-01Typedclean
Instantyesterday i go to the office and i see the new design
Lightyesterday i go to the office and i see the new design 339 msthe cleanup was rejected, so the original was kept
PolishYesterday, I go went to the office and i see saw the new design. 348 ms
num-chapterTypedclean
Instantlet's start with chapter one
LightLet's start with chapter one. 238 ms
PolishLet's start with chapter one. 194 ms
num-currencyTypedclean
Instantthe plan costs $25 a month
LightThe plan costs $25 a month. 232 ms
PolishThe plan costs $25 a month. 230 ms
num-decimalTypedclean
Instantthe download is 3 point 5 gigabytes
LightThe download is 3 point 5 3.5 gigabytes. 258 ms
PolishThe download is 3 point 5 3.5 gigabytes. 257 ms
num-decimal-pctTypedclean
Instantwe hit 99 point 9% uptime
LightWe hit 99 point 9% 99.9% uptime. 278 ms
PolishWe hit 99 point 9% 99.9% uptime. 291 ms
num-first-placeTypedclean
Instantshe came in first place at the meet
LightShe came in first place at the meet. 257 ms
PolishShe came in first place at the meet. 231 ms
num-halfTypedclean
Instantgive me half a second to think
LightGive me half a second to think. 253 ms
PolishGive me half a second to think. 213 ms
num-largeTypedclean
Instantwe shipped 120 units
LightWe shipped 120 units. 215 ms
PolishWe shipped 120 units. 209 ms
num-no-oneTypedclean
Instantno one showed up to the review
LightNo one showed up to the review. 215 ms
PolishNo one showed up to for the review. 210 ms