New method adapts speech recognition AI using only text data

Adapting large language models to handle automatic speech recognition (ASR) in specialized fields has been a tricky problem—especially when only text examples are available. A new research paper from arXiv offers a solution that sidesteps many of the pitfalls that plague conventional approaches.

The core challenge#

When researchers try to fine-tune the language component of an LLM-based speech system using domain-specific text alone, they typically run into trouble. The model learns new vocabulary and patterns from the text, but in doing so, it can degrade the delicate connection between what the system learns from speech signals and what it learns from text—the alignment that makes the whole system work. This mismatch hurts overall performance.

A denoising-based solution#

The team proposed reframing text-only adaptation as a text denoising task. Rather than simply adjusting the model to match target-domain text, they train the LLM to clean up noisy or corrupted transcripts and recover the original, correct version. This indirect approach preserves the audio-to-text alignment while still absorbing domain-specific patterns from the text data.

The method requires no architectural tweaks or additional parameters—it's a lightweight modification that works within existing systems.

Results#

Testing across two datasets yielded improvements of up to 22.1% relative gain compared to earlier text-only adaptation methods, establishing a new reference point for the field.