← All runs

Evaluation run

audio run (incl LibriSpeech)

6 July 2026, 11:18 · eval-engine · 54c20aa · 140 cases, 140 spoken

Did it hear the words right?

Clean human speech20 clips · real recordings3.4%
Bluetooth headset15 clips · HFP, the worst microphone people use16.3%
Synthetic voice89 clips · generated speech19.2%
Noisy room16 clips · speech mixed with room noise23.5%

Did it clean up correctly and safely?

51%
Light, ships on by default
72/140 cases passed · 249 ms typical
40%
Polish, experimental
48/120 cases passed · 267 ms typical
When a case fails, whose fault is it?
105 the microphone misheard the words  ·  35 the cleanup itself slipped

Every case

Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.

140 cases match

ls-1272-135031-0000Real speechclean
HeardBecause you were are sleeping instead of conquering, the lovely Rose Princess has become a fiddle without a bow, while poor Shaggy sits there a cooing dove. 4% misheard
Was saidbecause you were sleeping instead of conquering the lovely rose princess has become a fiddle without a bow while poor shaggy sits there a cooing dove
InstantBecause you are sleeping instead of conquering, the lovely Rose Princess has become a fiddle without a bow, while poor Shaggy sits there a cooing dove.
LightBecause you are sleeping instead of conquering, the lovely Rose Princess has become a fiddle without a bow, while poor Shaggy sits there a cooing dove. 753 ms
ls-1272-135031-0001Real speechclean
HeardHe has gone, and gone for good, answered Polychrome, who had managed to squeeze into the room beside the dragon, and had witnessed the occurrences with much interest.. 0% misheard
Was saidhe has gone and gone for good answered polychrome who had managed to squeeze into the room beside the dragon and had witnessed the occurrences with much interest
InstantHe has gone, and gone for good, answered Polychrome, who had managed to squeeze into the room beside the dragon, and had witnessed the occurrences with much interest..
LightHe has gone, and gone for good, answered Polychrome, who had managed to squeeze into the room beside the dragon, and had witnessed the occurrences with much interest. 749 ms
ls-1272-135031-0002Real speechclean
HeardI have remained a prisoner only because I wished to be one. And with this he stepped forward and burst the stout chains as easily as if they had been threads. 0% misheard
Was saidi have remained a prisoner only because i wished to be one and with this he stepped forward and burst the stout chains as easily as if they had been threads
InstantI have remained a prisoner only because I wished to be one. And with this he stepped forward and burst the stout chains as easily as if they had been threads.
LightI have remained a prisoner only because I wished to be one. And with this he stepped forward and burst the stout chains as easily as if they had been threads. 764 ms
ls-1272-135031-0003Real speechclean
HeardThe little girl had been asleep, but she heard the raps and opened the door. 0% misheard
Was saidthe little girl had been asleep but she heard the raps and opened the door
InstantThe little girl had been asleep, but she heard the raps and opened the door.
LightThe little girl had been asleep, but she heard the raps and opened the door. 516 ms
ls-1272-135031-0004Real speechclean
HeardThe king has fled in disgrace, and your friends are asking for you. 0% misheard
Was saidthe king has fled in disgrace and your friends are asking for you
InstantThe king has fled in disgrace, and your friends are asking for you.
LightThe king has fled in disgrace, and your friends are asking for you. 372 ms
noisy-corr-money-noNoisyneeds review
HeardThe plan is twenty five 35 no forty dollars $40 a month. 40% misheard
Was saidthe plan is twenty five no forty dollars a month
InstantThe plan is 35 no $40 a month.
LightThe plan is 35 no $40 a month. 236 ms
PolishThe plan is $35 no $40 a month, not $40. 332 ms
Why?the microphone misheard 40% of the words
noisy-corr-name-chainNoisyneeds review
HeardCall John no Jane no actually Mike. 0% misheard
Was saidcall john no jane no actually mike
InstantCall John no Jane no actually Mike.
LightCall John, no Jane, no actually Mike. 247 ms
PolishCall John, no not Jane. no It's actually Mike. 277 ms
Why?the result still contains “John” and “Jane”, which this case forbids
noisy-corr-num-chainNoisyneeds review
HeardWe need twenty 20 no thirty 30 no forty 40 units. 38% misheard
Was saidwe need twenty no thirty no forty units
InstantWe need 20 no 30 no 40 units.
LightWe need 20, no 30, no and 40 units. 355 ms
PolishWe need 20, no 30, no 40 units. 385 ms
Why?the result still contains “20” and “30”, which this case forbids; the microphone misheard 38% of the words
noisy-corr-time-noNoisyneeds review
HeardLet's meet at three 3 no four thirty 4.30. 43% misheard
Was saidlet's meet at three no four thirty
InstantLet's meet at 3 no 4.30.
LightLet's meet at 3 no 4:30. 255 ms
PolishLet's meet at 3 no 4:30. 264 ms
Why?the microphone misheard 43% of the words
noisy-disf-dayNoisyclean
HeardLet's do it on Monday sorry I mean Tuesday. 0% misheard
Was saidlet's do it on monday sorry i mean tuesday
InstantLet's do it on Monday sorry I mean Tuesday.
LightLet's do it on Monday sorry I mean Tuesday. 193 ms
PolishLet's do it on Monday sorry I mean Tuesday. 198 ms