Bluetooth headset15 clips · HFP, the worst microphone people use16.3%
Synthetic voice89 clips · generated speech19.2%
Noisy room16 clips · speech mixed with room noise23.5%
Did it clean up correctly and safely?
55%
Light, ships on by default
77/140 cases passed · 255 ms typical
49%
Polish, experimental
59/120 cases passed · 273 ms typical
When a case fails, whose fault is it?
105 the microphone misheard the words · 19 the cleanup itself slipped
Every case
Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.
140 cases match
tts-faith-codeSyntheticclean
HeardWrite a Python function to reverse a string. 0% misheard
Was saidwrite a python function to reverse a string
InstantWrite a Python function to reverse a string.
LightWrite a Python function to reverse a string. 219 ms
PolishWrite a Python function to reverse a string. 304 msthe cleanup was rejected, so the original was kept
tts-faith-code-bashSyntheticclean
HeardGive me a bash command to delete node modules. 0% misheard
Was saidgive me a bash command to delete node modules
InstantGive me a bash command to delete node modules.
LightGive me a bash command to delete node modules. 262 ms
PolishGive me a bash command to delete node modules. 524 msthe cleanup was rejected, so the original was kept
tts-faith-command-deleteSyntheticclean
HeardDelete all the files in my downloads folder. 0% misheard
Was saiddelete all the files in my downloads folder
InstantDelete all the files in my downloads folder.
LightDelete all the files in my downloads folder. 239 ms
PolishDelete all the files in my downloads folder. 442 msthe cleanup was rejected, so the original was kept
tts-faith-countSyntheticneeds review
HeardCount from one 1 to five 5 for me. 29% misheard
Was saidcount from one to five for me
InstantCount from 1 to 5 for me.
LightCount from 1 to 5 for me. 256 ms
PolishCount from 1 to 5 for me. 338 msthe cleanup was rejected, so the original was kept
Why?the result is missing “count from one to five”; the microphone misheard 29% of the words
tts-faith-defineSyntheticclean
HeardWhat does ubiquitous mean? 0% misheard
Was saidwhat does ubiquitous mean
InstantWhat does ubiquitous mean?
LightWhat does ubiquitous mean? 154 ms
PolishWhat does "ubiquitous" mean? 210 ms
tts-faith-define-2Syntheticclean
HeardDefine the word serendipity? 0% misheard
Was saiddefine the word serendipity
InstantDefine the word serendipity?
LightDefine the word serendipity? 218 ms
PolishDefine the word serendipity? 478 msthe cleanup was rejected, so the original was kept
tts-faith-email-composeSyntheticclean
HeardWrite an email to my boss telling him I need next Friday off. 0% misheard
Was saidwrite an email to my boss telling him i need next friday off
InstantWrite an email to my boss telling him I need next Friday off.
LightWrite an email to my boss telling him I need next Friday off. 870 msthe cleanup was rejected, so the original was kept
PolishWrite an email to my boss telling him I need next Friday off. 1028 msthe cleanup was rejected, so the original was kept
tts-faith-fact-moonSyntheticclean
HeardHow far is the moon from the earth? 0% misheard
Was saidhow far is the moon from the earth
InstantHow far is the moon from the earth?
LightHow far is the moon from the earth? 220 ms
PolishHow far is the moon from the Earth? 213 ms
tts-faith-fact-popSyntheticneeds review
HeardWhat's the population of Japan? 0% misheard
Was saidwhat's the population of japan
InstantWhat's the population of Japan?
LightWhat's the population of Japan? 200 ms
PolishWhat's The population of Japan is approximately 126 million people. 317 ms
Why polish?the result still contains “million” and “126”, which this case forbids
tts-faith-fact-questionSyntheticclean
HeardWho was the first President of the United States? 0% misheard
Was saidwho was the first president of the united states
InstantWho was the first President of the United States?
LightWho was the first President of the United States? 246 ms
PolishWho was the first President of the United States? 236 ms