Bluetooth headset15 clips · HFP, the worst microphone people use16.3%
Synthetic voice89 clips · generated speech19.2%
Noisy room16 clips · speech mixed with room noise23.5%
Did it clean up correctly and safely?
55%
Light, ships on by default
77/140 cases passed · 288 ms typical
49%
Polish, experimental
59/120 cases passed · 273 ms typical
When a case fails, whose fault is it?
105 the microphone misheard the words · 19 the cleanup itself slipped
Every case
Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.
140 cases match
tts-faith-codeSyntheticclean
HeardWrite a Python function to reverse a string. 0% misheard
Was saidwrite a python function to reverse a string
InstantWrite a Python function to reverse a string.
LightWrite a Python function to reverse a string. 229 ms
PolishWrite a Python function to reverse a string. 303 msthe cleanup was rejected, so the original was kept
tts-faith-code-bashSyntheticclean
HeardGive me a bash command to delete node modules. 0% misheard
Was saidgive me a bash command to delete node modules
InstantGive me a bash command to delete node modules.
LightGive me a bash command to delete node modules. 336 ms
PolishGive me a bash command to delete node modules. 504 msthe cleanup was rejected, so the original was kept
tts-faith-command-deleteSyntheticclean
HeardDelete all the files in my downloads folder. 0% misheard
Was saiddelete all the files in my downloads folder
InstantDelete all the files in my downloads folder.
LightDelete all the files in my downloads folder. 269 ms
PolishDelete all the files in my downloads folder. 544 msthe cleanup was rejected, so the original was kept
tts-faith-countSyntheticneeds review
HeardCount from one 1 to five 5 for me. 29% misheard
Was saidcount from one to five for me
InstantCount from 1 to 5 for me.
LightCount from 1 to 5 for me. 292 ms
PolishCount from 1 to 5 for me. 342 msthe cleanup was rejected, so the original was kept
Why?the result is missing “count from one to five”; the microphone misheard 29% of the words
tts-faith-defineSyntheticclean
HeardWhat does ubiquitous mean? 0% misheard
Was saidwhat does ubiquitous mean
InstantWhat does ubiquitous mean?
LightWhat does ubiquitous mean? 165 ms
PolishWhat does "ubiquitous" mean? 218 ms
tts-faith-define-2Syntheticclean
HeardDefine the word serendipity? 0% misheard
Was saiddefine the word serendipity
InstantDefine the word serendipity?
LightDefine the word serendipity? 256 ms
PolishDefine the word serendipity? 560 msthe cleanup was rejected, so the original was kept
tts-faith-email-composeSyntheticclean
HeardWrite an email to my boss telling him I need next Friday off. 0% misheard
Was saidwrite an email to my boss telling him i need next friday off
InstantWrite an email to my boss telling him I need next Friday off.
LightWrite an email to my boss telling him I need next Friday off. 1004 msthe cleanup was rejected, so the original was kept
PolishWrite an email to my boss telling him I need next Friday off. 1098 msthe cleanup was rejected, so the original was kept
tts-faith-fact-moonSyntheticclean
HeardHow far is the moon from the earth? 0% misheard
Was saidhow far is the moon from the earth
InstantHow far is the moon from the earth?
LightHow far is the moon from the earth? 233 ms
PolishHow far is the moon from the Earth? 296 ms
tts-faith-fact-popSyntheticneeds review
HeardWhat's the population of Japan? 0% misheard
Was saidwhat's the population of japan
InstantWhat's the population of Japan?
LightWhat's the population of Japan? 251 ms
PolishWhat's The population of Japan is approximately 126 million people. 303 ms
Why polish?the result still contains “million” and “126”, which this case forbids
tts-faith-fact-questionSyntheticclean
HeardWho was the first President of the United States? 0% misheard
Was saidwho was the first president of the united states
InstantWho was the first President of the United States?
LightWho was the first President of the United States? 287 ms
PolishWho was the first President of the United States? 237 ms