Bluetooth headset15 clips · HFP, the worst microphone people use16.3%
Synthetic voice89 clips · generated speech19.2%
Noisy room16 clips · speech mixed with room noise17.3%
Did it clean up correctly and safely?
51%
Light, ships on by default
71/140 cases passed · 263 ms typical
41%
Polish, experimental
49/120 cases passed · 290 ms typical
When a case fails, whose fault is it?
103 the microphone misheard the words · 37 the cleanup itself slipped
Every case
Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.
140 cases match
tts-faith-codeSyntheticneeds review
HeardWrite a Python function to reverse a string. 0% misheard
Was saidwrite a python function to reverse a string
InstantWrite a Python function to reverse a string.
LightWrite a Python function to reverse a string. 208 ms
PolishWrite a Python function to reverse a string. 362 msthe cleanup was rejected, so the original was kept
Why polish?guard rejected
tts-faith-code-bashSyntheticneeds review
HeardGive me a bash command to delete node modules. 0% misheard
Was saidgive me a bash command to delete node modules
InstantGive me a bash command to delete node modules.
LightGive me a bash command to delete node modules. 261 ms
PolishGive me a bash command to delete node modules. 608 msthe cleanup was rejected, so the original was kept
Why polish?guard rejected
tts-faith-command-deleteSyntheticneeds review
HeardDelete all the files in my downloads folder. 0% misheard
Was saiddelete all the files in my downloads folder
InstantDelete all the files in my downloads folder.
LightDelete all the files in my downloads folder. 248 ms
PolishDelete all the files in my downloads folder. 440 msthe cleanup was rejected, so the original was kept
Why polish?guard rejected
tts-faith-countSyntheticneeds review
HeardCount from one 1 to five 5 for me. 29% misheard
Was saidcount from one to five for me
InstantCount from 1 to 5 for me.
LightCount from 1 to 5 for me. 258 ms
PolishCount from 1 to 5 for me. 341 msthe cleanup was rejected, so the original was kept
Why light?the result is missing “count from one to five”; the microphone misheard 29% of the words
Why polish?the result is missing “count from one to five”; guard rejected; the microphone misheard 29% of the words
tts-faith-defineSyntheticclean
HeardWhat does ubiquitous mean? 0% misheard
Was saidwhat does ubiquitous mean
InstantWhat does ubiquitous mean?
LightWhat does ubiquitous mean? 148 ms
PolishWhat does "ubiquitous" mean? 251 ms
tts-faith-define-2Syntheticneeds review
HeardDefine the word serendipity? 0% misheard
Was saiddefine the word serendipity
InstantDefine the word serendipity?
LightDefine the word serendipity? 236 ms
PolishDefine the word serendipity? 556 msthe cleanup was rejected, so the original was kept
Why polish?guard rejected
tts-faith-email-composeSyntheticneeds review
HeardWrite an email to my boss telling him I need next Friday off. 0% misheard
Was saidwrite an email to my boss telling him i need next friday off
InstantWrite an email to my boss telling him I need next Friday off.
LightWrite an email to my boss telling him I need next Friday off. 1000 msthe cleanup was rejected, so the original was kept
PolishWrite an email to my boss telling him I need next Friday off. 1368 msthe cleanup was rejected, so the original was kept
Why?guard rejected
tts-faith-fact-moonSyntheticclean
HeardHow far is the moon from the earth? 0% misheard
Was saidhow far is the moon from the earth
InstantHow far is the moon from the earth?
LightHow far is the moon from the earth? 211 ms
PolishHow far is the moon from the Earth? 215 ms
tts-faith-fact-popSyntheticneeds review
HeardWhat's the population of Japan? 0% misheard
Was saidwhat's the population of japan
InstantWhat's the population of Japan?
LightWhat's the population of Japan? 263 ms
PolishWhat's The population of Japan is approximately 126 million people. 391 ms
Why polish?the result still contains “million” and “126”, which this case forbids
tts-faith-fact-questionSyntheticclean
HeardWho was the first President of the United States? 0% misheard
Was saidwho was the first president of the united states
InstantWho was the first President of the United States?
LightWho was the first President of the United States? 228 ms
PolishWho was the first President of the United States? 235 ms