Bluetooth headset15 clips · HFP, the worst microphone people use16.3%
Synthetic voice89 clips · generated speech19.2%
Noisy room16 clips · speech mixed with room noise23.5%
Did it clean up correctly and safely?
55%
Light, ships on by default
77/140 cases passed · 288 ms typical
49%
Polish, experimental
59/120 cases passed · 273 ms typical
When a case fails, whose fault is it?
105 the microphone misheard the words · 19 the cleanup itself slipped
Every case
Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.
140 cases match
noisy-real-noteNoisyneeds review
HeardTodo for tomorrow finish the report call the bank and pick up groceries. Todo for tomorrow finish the report call the bank and pick up groceries. 100% misheard
Was saidtodo for tomorrow finish the report call the bank and pick up groceries
InstantTodo for tomorrow finish the report call the bank and pick up groceries. Todo for tomorrow finish the report call the bank and pick up groceries.
LightTodo for tomorrow: finish the report, call the bank, and pick up groceries. Todo for tomorrow finish the report call the bank and pick up groceries. 418 ms
PolishTodo for tomorrow: finish the report, call the bank, and pick up groceries. Todo for tomorrow finish the report call the bank and pick up groceries. 467 ms
Why?the microphone misheard 100% of the words
tts-caps-properSyntheticneeds review
HeardI met sarah Sarai in New York on Tuesday. 13% misheard
Was saidi met sarah in new york on tuesday
InstantI met Sarai in New York on Tuesday.
LightI met Sarai in New York on Tuesday. 293 ms
PolishI met Sarai in New York on Tuesday. 247 ms
Why?the result is missing “Sarah”
tts-corr-money-noSyntheticneeds review
HeardThe plan is twenty five 35 no forty dollars $40 a month. 40% misheard
Was saidthe plan is twenty five no forty dollars a month
InstantThe plan is 35 no $40 a month.
LightThe plan is 35 no $40 a month. 335 ms
PolishThe plan is $35 no $40 a month, not $40. 323 ms
Why?the microphone misheard 40% of the words
tts-corr-name-chainSyntheticneeds review
HeardCall John no Jane no actually Mike. 0% misheard
Was saidcall john no jane no actually mike
InstantCall John no Jane no actually Mike.
LightCall John no Jane no actually Mike. 182 msthe cleanup was rejected, so the original was kept
PolishCall John no Jane no actually Mike. 139 msthe cleanup was rejected, so the original was kept
Why?the result still contains “John” and “Jane”, which this case forbids
tts-corr-num-chainSyntheticneeds review
HeardWe need twenty 20 no thirty 30 no forty 40 units. 38% misheard
Was saidwe need twenty no thirty no forty units
InstantWe need 20 no 30 no 40 units.
LightWe need 20 no 30 no 40 units. 231 ms
PolishWe need 20, no 30, no and 40 units. 356 ms
Why light?the result is missing “40”; the result still contains “30”, which this case forbids; the microphone misheard 38% of the words
Why polish?the result still contains “20” and “30”, which this case forbids; the microphone misheard 38% of the words
tts-corr-time-noSyntheticneeds review
HeardLet's meet at three 3 no four thirty 4.30. 43% misheard
Was saidlet's meet at three no four thirty
InstantLet's meet at 3 no 4.30.
LightLet's meet at 3 no 4:30. 314 ms
PolishLet's meet at 3 no 4.30. 3:30. 256 ms
Why light?the microphone misheard 43% of the words
Why polish?the result is missing “4:30”; the result still contains “3:3”, which this case forbids; the microphone misheard 43% of the words
tts-disf-daySyntheticclean
HeardLet's do it on Monday sorry I mean Tuesday. 0% misheard
Was saidlet's do it on monday sorry i mean tuesday
InstantLet's do it on Monday sorry I mean Tuesday.
LightLet's do it on Monday sorry I mean Tuesday. 213 ms
PolishLet's do it on Monday sorry I mean Tuesday. 195 ms
tts-disf-false-startsSyntheticneeds review
HeardSo the the main me thing is uh the main that my point is we need to ship. 33% misheard
Was saidso the the main thing is uh the main point is we need to ship
InstantSo the me thing is that my point is we need to ship.
LightSo the me thing is that my point is we need to ship. 389 ms
PolishSo the me thing is that my point is that we need to ship. 424 ms
Why?the microphone misheard 33% of the words
tts-disf-heavy-stutterSyntheticclean
HeardI I I think we we should just go go now. 0% misheard
Was saidi i i think we we should just go go now
InstantI I I think we we should just go go now.
LightI I I think we we should just go go now. 252 ms
PolishI I I think we we should just go go now. 266 ms
tts-disf-repeatSyntheticneeds review
Heardthe Thother the report is is ready to go. 13% misheard
Was saidthe the report is is ready to go
InstantThother the report is is ready to go.
LightThother the report is is ready to go. 227 msthe cleanup was rejected, so the original was kept
PolishThother The report is is ready to go. 196 ms
Why light?the result is missing “report is ready”; the result still contains “is is”, which this case forbids