Survey Maps the Growing Field of Automated Reinforcement Learning

Reinforcement learning and its deep-learning variants have become standard tools for tackling sequential decision-making problems. Yet building an effective RL system remains labor-intensive: practitioners must make countless choices about how to model a problem, which algorithm to deploy, and how to configure its many tuning parameters. Even small changes to these settings can dramatically alter performance.

The Automation Challenge#

This burden of manual engineering has historically fallen to RL specialists. But as reinforcement learning expands into domains like combinatorial optimization—where the primary experts may lack deep RL training—the need to automate these decisions has grown urgent. Decisions that currently require manual intervention include defining state and action spaces, selecting batch sizes, setting update frequencies, and determining time steps for learning cycles.

What AutoRL Encompasses#

Automated Reinforcement Learning (AutoRL) tackles this challenge by automating several key components of the RL pipeline: MDP (Markov Decision Process) modeling, algorithm selection, and hyperparameter optimization. Rather than asking practitioners to hand-tune each element, AutoRL frameworks aim to discover good configurations automatically, making RL accessible to researchers and engineers without specialist knowledge.

Recent Developments#

Recent work in this space has incorporated large language models (LLMs) as a novel approach to automating RL design choices. Beyond LLM-based methods, the field is also exploring complementary techniques that, while not yet purpose-built for AutoRL, show promise for future integration into automated workflows.

Open Questions#

The survey identifies ongoing challenges and unresolved questions in AutoRL research, laying out potential directions for the field to address as automation becomes more central to how reinforcement learning systems are built and deployed.