Researchers Propose Smart Router to Select Best AI Model for Each Prompt's Diversity

When a prompt invites multiple valid answers, the challenge lies in getting an AI system to produce the full spectrum of reasonable responses rather than converging on a single answer. Researchers have found that different language models excel at different types of diversity tasks, suggesting a fresh approach to the problem.

The Diversity Challenge#

Language models often generate high-quality but narrow responses to open-ended prompts. While individual models perform well, no single model consistently outperforms others across diverse prompting scenarios. The researchers evaluated 18 different large language models and confirmed this pattern: each model dominates on certain prompts while lagging on others, even though every prompt has at least one model that substantially outperforms its peers at generating varied, valid responses.

A New Evaluation Framework#

The team introduced diversity coverage, a metric that assesses how well a model captures the full range of acceptable answers. Rather than measuring just quality or quantity in isolation, this metric evaluates each unique answer's quality and compares the predicted answer set against the theoretically best possible set of the same size. This allows for standardized comparison of how comprehensively different models handle diversity.

Routing to the Right Model#

Rather than settle on one best-all model, the researchers trained a router system that predicts which language model performs best for each specific query. On the NB-WildChat dataset, this adaptive routing approach achieved a diversity coverage of 26.3%, compared to 23.8% for relying on a single strongest baseline model—a meaningful improvement that could better serve users with varied information needs.

The approach also demonstrated generalization to unseen data in the NB-Curated dataset and proved robust across different prompting strategies for answer generation.

Implications#

The work opens a direction for systems that leverage multiple specialized models rather than betting everything on a single architecture. When diverse outputs matter—in brainstorming, research synthesis, or any task requiring comprehensive coverage of possibilities—dynamic model selection offers a practical way to expand the range of generated responses.