New Chinese Speech Recognition System Corrects Phonetic Errors While Maintaining Output Length

A team of researchers has introduced PERL, a novel approach to correcting errors in Chinese automatic speech recognition (ASR) systems that grapples with the unique challenge of phonetic confusion—where characters that sound similar get mixed up—while also ensuring the corrected output matches the expected length.

The Core Challenge#

Fixing transcription mistakes in Chinese is particularly tricky because many characters share identical or near-identical pronunciations (encoded in their Pinyin romanizations). Traditional correction methods either treat Pinyin as secondary information in prompts or features, or they use open-ended generation that can produce outputs of unpredictable length. Neither approach fully solves the problem of maintaining both accuracy and length consistency when working with multiple competing ASR hypotheses.

How PERL Works#

The PERL system combines two key innovations. First, it predicts the exact length of the corrected output and enforces that constraint through a budgeting mechanism during generation. Second, it blends two types of information—semantic understanding and phonetic patterns (Pinyin)—by using token-level gates that adapt based on the overall sentence context. This allows the model to switch between relying on word meaning and sound-based cues depending on what the situation demands.

Test Results#

Experiments on established benchmarks and a new domain-specific dataset called DoAD demonstrate significant improvements. PERL reduced character error rates by 29.11% on Aishell-1 and achieved reductions up to approximately 70% on the DoAD benchmark, all while keeping processing latency low. Analysis of the results reveals when and why the model leans on phonetic information versus strict semantic constraints, offering insights into how the two knowledge sources interact during correction.

The work is documented in detail on arXiv.