A Kaldi phoneme comparison method checks TTS pronunciation errors

What happened

TTS often mispronounces polyphonic and rare Chinese characters, and manual listening is unreliable and does not scale. Baoyu shares a method that runs TTS audio through Kaldi, reads the phoneme sequence along its decoding path instead of the recognized Chinese characters, and compares it with the target characters’ correct pronunciations. https://twitter.com/goldengrape/status/2113110528541520206

Read next

• More changes in today’s briefingMicrosoft Research and Collaborator Resolve the KLS Conjecture With Human-Directed AI

The story so far

Baoyu (@dotey) shared a method for checking text-to-speech (TTS) pronunciation by using Kaldi to extract phonemes from generated audio and comparing them with the intended pronunciations of Chinese characters. The method reads the phoneme sequence along Kaldi’s decoding path rather than relying on the Chinese characters it recognizes. Baoyu presented the approach as a way to identify pronunciation errors involving rare Chinese characters and characters with multiple pronunciations, saying manual listening is unreliable and does not scale. Comparing phonemes individually with the target pronunciations is intended to establish which sounds the TTS system actually produced.