New framework tailors large language models to individual user preferences using natural language summaries

Most large language models are trained to satisfy broad population preferences, overlooking the fact that individuals have distinctly different needs and tastes. Researchers have proposed POPI, a new personalization system designed to adapt LLMs to unique user preferences at scale.

How it works#

POPI splits personalization into two interconnected parts linked through natural language: an inference component that converts scattered user signals into a succinct, human-readable preference profile, and a generation component that uses this profile to shape its outputs. Both parts train together using a unified optimization process, with reinforcement learning managing the inference step since it cannot be differentiated mathematically.

The key innovation is that learned preference summaries—once created for a user—remain valid across different generators. This means they can work with frozen, proprietary models and commercial APIs, not just models that the system has direct control over.

Performance gains#

Testing across four personalization benchmarks revealed that POPI generally delivers stronger personalization results. More remarkably, it achieves these improvements while slashing the amount of contextual information needed by as much as tenfold.

The unified optimization objective reveals an elegant interplay: it simultaneously minimizes errors in generating personalized responses and maximizes the informativeness of user summaries, creating a single loss function that drives both tasks forward.

Why it matters#

By encoding user preferences as natural language rather than embeddings or hidden representations, the approach gains interpretability and cross-model portability. Users and developers can inspect what the system has learned about them, and preference profiles can migrate to new tools as AI systems evolve.