LLMs Learn to Design Physical Machines Through Simulation and Feedback

Large language models excel at code generation in abstract, symbolic environments—but can they design machines that actually work? A new research effort explores this question by grounding program synthesis in physics.

A New Kind of Design Problem#

Researchers have introduced compositional machine design, a framework where LLMs generate machine programs that compose standardized mechanical parts. Rather than testing code in digital logic alone, success depends on how well the resulting machine performs in physics simulation. The team built BesiegeField, a testbed based on the machine-building video game Besiege, where agents convert textual functional specifications into working mechanical designs.

How the Benchmark Works#

In the system, LLM agents receive a functional requirement in natural language, then generate a machine program. The program is executed in simulation, and the agent receives quantitative rewards and state feedback based on physical performance. The researchers tested various approaches:

• Single-pass machine generation from text
• Iterative editing workflows where agents revise designs based on simulation results
• Hierarchical methods that decompose design into substeps

Models were benchmarked across representative machine-design tasks, with human performance measured as a reference point for task difficulty.

What Works—and What Doesn't#

Strong language models can recover task-relevant structures and occasionally achieve meaningful physical performance. However, they frequently stumble with spatially precise assembly, multi-component mechanism planning, and translating feedback into actionable structural modifications. These limitations reveal gaps between code generation (where LLMs shine) and embodied design (where spatial reasoning matters).

Reinforcement Learning Improves Results#

The team finetuned Qwen2.5-14B, an open-source LLM, using reinforcement learning trained on simulation-derived rewards. Under a fixed computational budget, RL improved the quality of machines discovered, suggesting that learning from physical feedback—rather than relying on pre-trained weights alone—helps bridge the gap between language and mechanics.