New benchmark reveals AI-generated web interfaces often fail in real-world use despite looking correct

Automated code generation by large language models can produce web interfaces that compile and appear functional in screenshots, yet falter when users actually interact with them. Researchers have introduced a new testing framework designed to catch these failures before they reach production.

The Gap Between Build Success and Functional Correctness#

Current approaches to evaluating AI-generated web user interface code rely on superficial checks: does the code compile? Does the rendered page look right? These static assessments miss a critical dimension—whether the interface actually works when people click buttons, fill forms, or otherwise engage with it. The problem is especially acute with complex interfaces like 3D simulations, where visual plausibility can mask broken physics or interaction logic.

Introducing WebUIProof#

A new benchmark called WebUIProof addresses this gap by testing web UI generation with executable interaction tests. Rather than stopping at code compilation and screenshots, the system runs dynamic tests in a headless browser environment. The benchmark covers two categories of interfaces: general-purpose web UIs (dashboards, games, interactive tools) and 3D interactive simulations (particle systems, physics engines).

The core innovation is a UI-agent harness that executes a plan-act-observe loop: it identifies DOM elements, performs user actions, records the resulting changes to the interface and underlying document structure, and verifies that specified assertions hold. This simulates real user workflows and captures failures that static checks would miss.

Testing Across Leading LLMs#

Evaluation across eight commercial large language models revealed that functional failures remain common even when interfaces render successfully. The problem is most pronounced with 3D simulation tasks, where rendering alone does not guarantee correct behavior. These findings suggest that current LLMs struggle to generate the precise, interactive logic required for complex web applications.

Improving Compact Models Through Reinforcement Learning#

The research also demonstrates that the UI-agent harness can generate training signals for improvement. Smaller models—such as Qwen2.5 14B and MIMO 7B—trained with reinforcement learning rewards derived from executable interaction tests show measurable gains in functional completion rates. Importantly, this targeted training also reduces build failures, suggesting that interaction-aware optimization benefits both robustness and correctness.