← All runs

Evaluation run

faithfulness scoring cleanup audio

6 July 2026, 13:38 · eval-engine · 232478e · 140 cases, 140 spoken

Did it hear the words right?

Clean human speech20 clips · real recordings3.4%
Bluetooth headset15 clips · HFP, the worst microphone people use16.3%
Synthetic voice89 clips · generated speech19.2%
Noisy room16 clips · speech mixed with room noise23.5%

Did it clean up correctly and safely?

55%
Light, ships on by default
77/140 cases passed · 255 ms typical
49%
Polish, experimental
59/120 cases passed · 273 ms typical
When a case fails, whose fault is it?
105 the microphone misheard the words  ·  19 the cleanup itself slipped

Every case

Each case shows the instant on-device cleanup, then what each mode did to it. Added words are marked, removed words are struck. For a spoken case, Heard is what the microphone transcribed and Was said is the reference it is scored against.

140 cases match

ls-1272-128104-0005Real speechclean
HeardIt is obviously unnecessary for us to point out how luminous these criticisms are, how delicate in expression. 0% misheard
Was saidit is obviously unnecessary for us to point out how luminous these criticisms are how delicate in expression
InstantIt is obviously unnecessary for us to point out how luminous these criticisms are, how delicate in expression.
LightIt is obviously unnecessary for us to point out how luminous these criticisms are, how delicate in expression. 500 ms
ls-1272-128104-0006Real speechclean
HeardOn the general principles of art, mister Quilter writes with equal lucidity. 0% misheard
Was saidon the general principles of art mister quilter writes with equal lucidity
InstantOn the general principles of art, mister Quilter writes with equal lucidity.
LightOn the general principles of art, Mister Quilter writes with equal lucidity. 382 ms
ls-1272-128104-0007Real speechclean
HeardPainting, he tells us, is of a different quality to mathematics, and finish in art is adding more fact. 0% misheard
Was saidpainting he tells us is of a different quality to mathematics and finish in art is adding more fact
InstantPainting, he tells us, is of a different quality to mathematics, and finish in art is adding more fact.
LightPainting, he tells us, is of a different quality to mathematics, and finish in art is adding more fact. 519 ms
ls-1272-128104-0008Real speechclean
HeardAs for etchings, they are of two kinds: British and foreign. 0% misheard
Was saidas for etchings they are of two kinds british and foreign
InstantAs for etchings, they are of 2 kinds: British and foreign.
LightAs for etchings, they are of 2 two kinds: British and foreign. 378 ms
ls-1272-128104-0009Real speechclean
HeardHe laments most bitterly the divorce that has been made between decorative art and what we usually call pictures, makes the a customary appeal to the last judgment, and reminds us that in the great days of art michael angelo Michelangelo was the furnishing upholsterer. 7% misheard
Was saidhe laments most bitterly the divorce that has been made between decorative art and what we usually call pictures makes the customary appeal to the last judgment and reminds us that in the great days of art michael angelo was the furnishing upholsterer
InstantHe laments most bitterly the divorce that has been made between decorative art and what we usually call pictures, makes a customary appeal to the last judgment, and reminds us that in the great days of art Michelangelo was the furnishing upholsterer.
LightHe laments most bitterly the divorce that has been made between decorative art and what we usually call pictures, makes a customary appeal to the last judgment, and reminds us that in the great days of art Michelangelo was the furnishing upholsterer. 1006 ms
ls-1272-128104-0010Real speechclean
HeardNear the fire, and the ornaments Fred brought home from India on the mantel mantle board. 7% misheard
Was saidnear the fire and the ornaments fred brought home from india on the mantel board
InstantNear the fire, and the ornaments Fred brought home from India on the mantle board.
LightNear the fire, and the ornaments Fred brought home from India on the mantle board. 420 msthe cleanup was rejected, so the original was kept
ls-1272-128104-0011Real speechclean
HeardIn fact, he is quite severe on mister Ruskin for not recognising that a picture should denote the frailty of man, and remarks, with pleasing courtesy and felicitous grace, that many phases of feeling 0% misheard
Was saidin fact he is quite severe on mister ruskin for not recognising that a picture should denote the frailty of man and remarks with pleasing courtesy and felicitous grace that many phases of feeling
InstantIn fact, he is quite severe on mister Ruskin for not recognising that a picture should denote the frailty of man, and remarks, with pleasing courtesy and felicitous grace, that many phases of feeling
LightIn fact, he is quite severe on Mister Ruskin for not recognising that a picture should denote the frailty of man, and remarks, with pleasing courtesy and felicitous grace, that many phases of feeling. 902 ms
ls-1272-128104-0012Real speechclean
HeardOnly, unfortunately, his own work never does get good. 0% misheard
Was saidonly unfortunately his own work never does get good
InstantOnly, unfortunately, his own work never does get good.
LightOnly, unfortunately, his own work never does get gets good. 303 ms
ls-1272-128104-0013Real speechclean
Heardmister Mr. Quilter has missed his chance, for he has failed even to make himself the tupper of painting. 6% misheard
Was saidmister quilter has missed his chance for he has failed even to make himself the tupper of painting
InstantMr. Quilter has missed his chance, for he has failed even to make himself the tupper of painting.
LightMr. Quilter has missed his chance, for he has failed even to make himself the tupper of painting. 487 msthe cleanup was rejected, so the original was kept
ls-1272-128104-0014Real speechneeds review
HeardBy Harry Quilter, m a MA 40% misheard
Was saidby harry quilter m a
InstantBy Harry Quilter, MA
LightBy Harry Quilter, MA 201 ms
Why?the microphone misheard 40% of the words