New Framework Boosts AI Translation for Mixed-Language Text by 68%

A team of researchers has introduced CHAI, a framework designed to substantially improve how large language models handle translation tasks involving code-mixed language—text that seamlessly blends multiple languages in conversation.

The Code-Mixing Challenge#

Code-mixed language is common in multilingual communities where speakers naturally weave together languages in everyday communication. Despite their general prowess, large language models have historically struggled with this type of mixed-language content. The research community has paid little attention to this gap, leaving LLMs comparatively weak at understanding and translating code-mixed text.

How CHAI Works#

The CHAI framework rests on four pillars. First, it taps LLMs themselves to generate annotations, sidestepping the traditional bottleneck of scarce, manually-created code-mixed datasets. Second, it converts these AI-generated annotations into large-scale preference data, then applies reinforcement learning from AI feedback (RLAIF)—a technique that uses AI judgments rather than human ones to refine model behavior. Third, it incorporates domain knowledge crafted by LLMs as guiding principles that allow the system to iteratively improve its responses. Fourth, the team conducted rigorous evaluations against real-world datasets and conditions.

Significant Performance Gains#

Testing revealed substantial improvements: CHAI-enhanced models outperformed leading open-source alternatives by an average of 68.45% in human-reviewed win rates on code-mixed translation benchmarks. This marks a meaningful step toward making inclusive, high-performing code-mixed language models freely available to researchers and developers.

The work underscores how reinforcement learning paired with AI feedback can tackle stubborn domain-specific weaknesses in LLMs, potentially opening doors for improvements in other underserved language tasks.