AI
New Benchmark Tests AI Agents at Circuit Layout Design Tasks
Researchers have created ICLayout-Bench, a standardized test suite that evaluates how well large language model agents perform at designing custom integrated circuit layouts using professional EDA tools.
A team of researchers has released ICLayout-Bench, a comprehensive benchmark designed to assess large language model agents on their capacity to design and optimize custom circuit layouts—a critical step in modern semiconductor manufacturing.
What the Benchmark Measures#
The benchmark places LLM agents in charge of using electronic design automation (EDA) software to build and refine GDSII layouts starting from transistor-level circuit specifications. Once agents complete their designs, the layouts undergo rigorous evaluation through physical verification, parasitic extraction, and post-layout simulation. Designs that successfully pass both physical and functional checks receive scores based on electrical performance and silicon area, measured on a 100-point scale.
Scope and Scale#
The evaluation framework encompasses 165 distinct design tasks spanning 13 different process design kits (PDKs) across technology nodes from 3 nanometers to 180 nanometers. The tasks cover a diverse range of circuit types—including analog, mixed-signal, and digital designs—and operate within both open-source and commercial EDA environments. Researchers tested 22 different model configurations representing seven distinct LLM variants on 21 public and 19 private tasks.
Key Findings#
Performance varied significantly across configurations. On publicly available tasks, mean layout scores ranged from 4.22 to 80.26 points; on private tasks, the range dropped to 1.43 to 67.01 points. Surprisingly, increasing computational effort through extended reasoning did not consistently yield better results. Additionally, configurations achieving similar scores often diverged substantially in their computational cost, suggesting opportunities for more efficient design approaches.
Why This Matters#
As AI systems increasingly take on specialized technical roles, benchmarking their performance in complex domains like circuit design provides essential insight into their readiness for real-world engineering applications. This work establishes a foundation for measuring progress in autonomous chip design automation.

