New method enables fine-grained pronunciation editing in Finnish speech synthesis

Researchers have developed a novel approach to synthesize second-language (L2) speech by directly editing native speech, rather than building entirely new recordings from scratch. The technique, called PPG2Speech, could significantly improve language learning applications, particularly for low-resourced languages where training data is scarce.

The challenge of low-resourced L2 speech synthesis#

Creating synthetic speech in a non-native accent has traditionally required large datasets of L2speakers. For languages with limited digital resources, this proves nearly impossible. Rather than building a speech system from the ground up, the researchers tackled a simpler problem: take existing native speech and modify it to sound like a non-native speaker learning the language.

How PPG2Speech works#

The model operates on Phonetic Posteriorgrams (PPGs)—a mathematical representation of speech sounds. By treating PPGs as an intermediate layer between acoustic features and meaning, the system can edit individual phonemes with surgical precision, without needing explicit text alignment. The approach builds on Matcha-TTS's flow-matching decoder, adding two refinements: Classifier-free Guidance to improve quality and Sway Sampling to enhance control during editing.

The system conditions its output on external speaker embeddings and pitch, preserving the original speaker's identity while adjusting pronunciation.

Validation on Finnish#

The team validated their method on Finnish—a language with relatively phonetic spelling and limited L2 speech datasets. Working with approximately 60 hours of training data, they compared PPG2Speech against traditional text-to-speech-based editing approaches. A newly proposed metric, Phonetic Aligned Consistency (PAC), measures how well the edited phonemes match the original intent.

Why it matters#

This approach addresses a genuine gap: language learners benefit from native-like L2 feedback, yet building systems for dozens of languages individually is impractical. By converting native speech rather than synthesizing from scratch, the method scales to lower-resourced contexts and opens a new avenue for personalized language instruction.