New Framework Lets Robots Learn Complex Tasks Through Generated Code Plans

A new approach to robotic long-horizon manipulation sidesteps the limitations of traditional learning-based methods by having large language models generate executable code plans. Rather than relying on dense feedback at every step or pre-trained action policies, the system — called DAHLIA — treats complex manipulation as episodic task planning.

How DAHLIA Works#

The framework uses progressively organized in-context examples to guide an LLM in synthesizing executable code from natural language instructions. This approach combines chain-of-thought reasoning with structured code generation, allowing the system to break down lengthy manipulation sequences into executable steps. A separate vision-language model serves as a task evaluator, assessing whether each episode succeeded and providing structured feedback for replanning when needed — without requiring per-step inference overhead.

Key Advantage: Robustness Without Dense Feedback#

Traditional approaches often demand step-by-step feedback or large labeled datasets, which limits their robustness and ability to generalize. DAHLIA's episodic evaluation strategy improves performance under partial observability — situations where the robot's sensors don't capture the full scene — and avoids the computational cost of continuous model inference.

Demonstrated Performance#

The system was tested on a variety of benchmarks and real-world tasks, including LoHoRavens, CALVIN, and Franka Kitchen environments. Across over 30 long-horizon manipulation tasks, DAHLIA showed strong performance and improved generalization to unseen scenarios, such as cluttered scenes and occluded objects that the system had not encountered during development.

Why It Matters#

This work suggests that code generation offers a promising alternative to traditional learned policies for robot manipulation, potentially making these systems more flexible and easier to adapt without massive amounts of training data.