disf-scratchTypedclean
Instantthe deadline is friday actually no let's say next monday
LightThe deadline is friday actually no let's say next Monday. 195 ms
PolishThe deadline is friday actually no let's say next Monday. 199 ms
Evaluation run
faithfulness scoring cleanup
Did it hear the words right?
This run graded typed text only, so there is nothing here to have misheard.
Did it clean up correctly and safely?
94%
Light, ships on by default
84/89 cases passed · 254 ms typical
89%
Polish, experimental
79/89 cases passed · 266 ms typical
When a case fails, whose fault is it?
0 the microphone misheard the words · 15 the cleanup itself slipped
Every case
Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.
89 cases match
disf-selfcorrect-nameTypedclean
Instantsend it to john i mean jane
LightSend it to john i mean Jane. 169 ms
PolishSend it to john i mean Jane. 174 ms
disf-selfcorrect-timeTypedclean
Instantlet's meet at 3 no wait 4:00
LightLet's meet at 3 no wait 4:00. 266 ms
PolishLet's meet at 3 no wait 4:00. 268 ms
edge-emailTypedclean
Instantemail me@john.smith at gmail dot com
Lightemail me@john.smith at gmail dot com me@john.smith@gmail.com 238 ms
Polishemail me@john.smith me at gmail dot com me@john.smith@gmail.com 276 ms
edge-gibberishTypedclean
Instantasdf qwerty test test one two three
Lightasdf qwerty test test one two three. 253 ms
Polishasdf qwerty. Test. test One two three. 269 ms
edge-mixedTypedclean
Instanthey so the the lyzr demo is at 4:30 no wait 5:00 and accuracy is like 90% now
LightHey, so the the Lyzr demo is at 4:30 no wait 5:00, and accuracy is like 90%. now 536 ms
PolishHey, so the the Lyzr demo is at 4:30 no wait 5:00, and accuracy is like 90%. now 640 ms
edge-shortTypedclean
Instantyeah sounds good
LightYeah, sounds good. 181 ms
PolishYeah, sounds good. 175 ms
edge-singleTypedclean
Instantperfect
LightPerfect. 130 ms
PolishPerfect. 125 ms
faith-already-cleanTypedclean
InstantThe quarterly report is due on Friday.
LightThe quarterly report is due on Friday. 223 ms
PolishThe quarterly report is due on Friday. 224 ms
faith-codeTypedclean
Instantwrite a python function to reverse a string
LightWrite a Python function to reverse a string. 248 ms
Polishwrite a python function to reverse a string 881 msthe cleanup was rejected, so the original was kept