Eloquence team tackles multilingual speech recognition with layered projector approach

The Eloquence team has released a study on their approach to multilingual conversational speech recognition, submitted as part of the Challenge and Workshop on Multilingual Conversational Speech Language Model (MLC-SLM). The work explores multiple paths to building more robust automatic speech recognition (ASR) systems that can handle real-world dialogue across different languages.

Three-pronged experimental strategy#

The team evaluated the official challenge baseline by training two different projector architectures—a linear projector and a QFormer—each paired with different foundation models. This comparison helped surface the strengths and limitations of the baseline approach. Beyond that, they developed a custom multilingual linear projector using the SLAM-ASR framework, aiming to create a more tailored solution for the multilingual setting.

Enhancing robustness through learning techniques#

A key focus was investigating how contrastive learning and expanded conversational context could strengthen recognition accuracy. By drawing on more dialogue history, the team sought to improve the system's ability to understand speaker intent and linguistic patterns—critical for real-world spoken dialogue systems that must handle the messy reality of how people actually talk.

Why it matters#

As conversational AI becomes increasingly deployed across regions and languages, building systems that perform reliably on authentic speech data—rather than clean, studio-recorded audio—is essential. This work contributes to that broader goal by experimenting with architectural choices and training strategies that could make multilingual ASR systems more practical and resilient.