New Algorithm Boosts LLM Exploration Without Model Retraining

Large language model agents designed for sequential decision-making often struggle with a critical limitation: they tend to repeat the same outputs rather than exploring diverse options. This narrow behavior leads to suboptimal solutions and agents getting trapped in loops. A new approach called DORA Explorer aims to address this problem without requiring any model retraining.

The Core Challenge#

Standard techniques for injecting variation into language model outputs—like adjusting temperature during sampling—work at the individual token level. This granular approach fails to encourage diversity where it actually matters: at the sequence level, where complete actions are generated. As a result, agents make the same decisions repeatedly, limiting their ability to navigate complex environments effectively.

How DORA Works#

DORA Explorer (Diversity-Oriented Ranking of Actions) operates at inference time, meaning it doesn't demand any training or fine-tuning of the underlying model. The algorithm generates multiple candidate actions, evaluates them using sequence-level log-probability statistics, and selects actions through a configurable exploration parameter. This allows practitioners to tune how adventurous the agent becomes.

Performance Results#

Researchers first validated the approach in the Multi-Armed Bandit setting, a foundational problem in decision theory, where DORA substantially outperformed temperature-based sampling. More comprehensive testing occurred in the Text Adventure Learning Environment Suite (TALES). On TextWorld tasks using the Qwen-2.5 7B model, the algorithm lifted performance from 31.43% under ReAct (a common prompting strategy) to 45.5%. Gains appeared consistent across different model families.

Beyond Exploration#

Beyond encouraging variety in action selection, DORA also prevents characteristic failure modes. Agents no longer get stuck in repetitive loops as readily, improving robustness in open-ended environments. Since the method is training-free, it can be applied immediately to existing deployed models.