Bluetooth headset15 clips · HFP, the worst microphone people use16.3%
Synthetic voice89 clips · generated speech19.2%
Noisy room16 clips · speech mixed with room noise23.5%
Did it clean up correctly and safely?
55%
Light, ships on by default
77/140 cases passed · 255 ms typical
49%
Polish, experimental
59/120 cases passed · 273 ms typical
When a case fails, whose fault is it?
105 the microphone misheard the words · 19 the cleanup itself slipped
Every case
Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.
140 cases match
noisy-real-noteNoisyneeds review
HeardTodo for tomorrow finish the report call the bank and pick up groceries. Todo for tomorrow finish the report call the bank and pick up groceries. 100% misheard
Was saidtodo for tomorrow finish the report call the bank and pick up groceries
InstantTodo for tomorrow finish the report call the bank and pick up groceries. Todo for tomorrow finish the report call the bank and pick up groceries.
LightTodo for tomorrow: finish the report, call the bank, and pick up groceries. Todo for tomorrow finish the report call the bank and pick up groceries. 392 ms
PolishTodo for tomorrow: finish the report, call the bank, and pick up groceries. Todo for tomorrow finish the report call the bank and pick up groceries. 405 ms
Why?the microphone misheard 100% of the words
tts-caps-properSyntheticneeds review
HeardI met sarah Sarai in New York on Tuesday. 13% misheard
Was saidi met sarah in new york on tuesday
InstantI met Sarai in New York on Tuesday.
LightI met Sarai in New York on Tuesday. 246 ms
PolishI met Sarai in New York on Tuesday. 285 ms
Why?the result is missing “Sarah”
tts-corr-money-noSyntheticneeds review
HeardThe plan is twenty five 35 no forty dollars $40 a month. 40% misheard
Was saidthe plan is twenty five no forty dollars a month
InstantThe plan is 35 no $40 a month.
LightThe plan is 35 no $40 a month. 240 ms
PolishThe plan is $35 no $40 a month, not $40. 329 ms
Why?the microphone misheard 40% of the words
tts-corr-name-chainSyntheticneeds review
HeardCall John no Jane no actually Mike. 0% misheard
Was saidcall john no jane no actually mike
InstantCall John no Jane no actually Mike.
LightCall John, no Jane, no actually Mike. 252 ms
PolishCall John, no not Jane. no It's actually Mike. 277 ms
Why?the result still contains “John” and “Jane”, which this case forbids
tts-corr-num-chainSyntheticneeds review
HeardWe need twenty 20 no thirty 30 no forty 40 units. 38% misheard
Was saidwe need twenty no thirty no forty units
InstantWe need 20 no 30 no 40 units.
LightWe need 20, no 30, no and 40 units. 356 ms
PolishWe need 20, no 30, no and 40 units. 364 ms
Why?the result still contains “20” and “30”, which this case forbids; the microphone misheard 38% of the words
tts-corr-time-noSyntheticneeds review
HeardLet's meet at three 3 no four thirty 4.30. 43% misheard
Was saidlet's meet at three no four thirty
InstantLet's meet at 3 no 4.30.
LightLet's meet at 3 no 4:30. 259 ms
PolishLet's meet at 3 no 4:30. 271 ms
Why?the microphone misheard 43% of the words
tts-disf-daySyntheticclean
HeardLet's do it on Monday sorry I mean Tuesday. 0% misheard
Was saidlet's do it on monday sorry i mean tuesday
InstantLet's do it on Monday sorry I mean Tuesday.
LightLet's do it on Monday sorry I mean Tuesday. 208 ms
PolishLet's do it on Monday sorry I mean Tuesday. 202 ms
tts-disf-false-startsSyntheticneeds review
HeardSo the the main me thing is uh the main that my point is we need to ship. 33% misheard
Was saidso the the main thing is uh the main point is we need to ship
InstantSo the me thing is that my point is we need to ship.
LightSo the me thing is that my point is we need to ship. 320 ms
PolishSo the me thing is that my point is that we need to ship. 337 ms
Why?the microphone misheard 33% of the words
tts-disf-heavy-stutterSyntheticclean
HeardI I I think we we should just go go now. 0% misheard
Was saidi i i think we we should just go go now
InstantI I I think we we should just go go now.
LightI I I think we we should just go go now. 234 ms
PolishI I I think we we should just go go now. 238 ms
tts-disf-repeatSyntheticneeds review
Heardthe Thother the report is is ready to go. 13% misheard
Was saidthe the report is is ready to go
InstantThother the report is is ready to go.
LightThother the report is is ready to go. 219 msthe cleanup was rejected, so the original was kept
PolishThother The report is is ready to go. 205 ms
Why light?the result is missing “report is ready”; the result still contains “is is”, which this case forbids