New System Learns User Preferences to Route Queries to the Right AI Model

Researchers have developed a new approach to directing user queries to the most suitable language model based on individual preferences, tackling a long-standing challenge in AI deployment: balancing quality outputs against the computational expense of running large models.

The Problem with One-Size-Fits-All Routing#

Traditional LLM routing systems treat all users the same, funneling identical questions to identical models regardless of who's asking. But users have distinct needs—some prioritize speed, others demand maximum accuracy, and still others care about specific model behaviors or styles. Existing personalization methods struggle because they either don't fully capture the nuance of how users and models interact, or they require large volumes of preference data that's often messy and incomplete.

How GMTRouter Works#

The new system, called GMTRouter, takes a graph-based approach to the problem. It represents the entire conversation history—spanning multiple exchanges between a user and various models—as a network with five interconnected node types: the user themselves, the different language models available, the queries submitted, the responses generated, and the individual conversation turns. This structure preserves the rich relational context that simple methods discard.

Rather than requiring massive retraining for each new user, GMTRouter employs a lightweight inductive learning framework combined with smart graph sampling techniques tailored to consider the user's position in the network. This allows it to extract preference patterns from just a handful of examples, making it practical for real-world deployment where user feedback is sparse.

Results and Practical Impact#

Testing demonstrated measurable improvements: the system achieved up to a 0.108 absolute gain in accuracy and 0.124 improvement in AUC over competing approaches. Crucially, GMTRouter proved capable of adapting to entirely new users using only a few examples, without requiring the extensive fine-tuning that would typically consume resources and time.

The code is publicly available, enabling other researchers and practitioners to build on the work and explore personalized routing in their own systems.