Multi-Agent Language Analysis Helps AI Identify Native Language More Reliably

Large language models frequently appear to perform well at identifying a person's native language by relying on surface-level hints—names, places, or stereotypes—rather than genuine linguistic markers of L1 background. Researchers have found that instructing models to ignore such shortcuts remains unreliable; misleading information can still sway predictions.

The Problem with Shortcuts#

Current LLM approaches to native language identification (NLI) tend to latch onto superficial contextual signals instead of deep linguistic patterns. When researchers have attempted to address this vulnerability by explicitly telling models to disregard misleading clues, the fix has proven inconsistent—the models continue to be swayed by false leads embedded in the text.

An Agent-Based Solution#

To strengthen the identification process, researchers have developed a new agentic pipeline modeled on forensic linguistics methodology. Rather than having a single language model make a judgment, the system deploys multiple specialized agents that each examine different types of linguistic evidence. These agents accumulate and categorize their findings independently, building a comprehensive evidentiary record.

A coordinating agent then reviews all the collected evidence and synthesizes it into a final native language prediction. By separating evidence gathering from judgment, and by grounding the approach in linguistic principles rather than contextual shortcuts, the pipeline becomes more resilient to misleading information.

Performance and Reliability#

Testing on two benchmark datasets demonstrated that the multi-agent approach significantly improves robustness against false contextual clues and produces more consistent predictions across different test conditions compared to standard prompting techniques.

This work draws on principles from forensic linguistics—where linguistic analysis aims to uncover authentic patterns beneath surface noise—to create an NLI system that prioritizes genuine language markers over incidental details.