AI
New Benchmark Tests AI Agents on Website Building Tasks
Vision2Web introduces a hierarchical evaluation framework for assessing how well language models and coding agents can build websites from design mockups.
A team of researchers has unveiled Vision2Web, a systematic evaluation framework that measures the performance of AI agents on website development tasks ranging from simple interface coding to complex full-stack projects.
What Vision2Web Measures#
The benchmark spans three escalating levels of difficulty: converting static designs to code, reproducing multi-page interactive websites, and building complete full-stack applications. The dataset comprises 193 distinct tasks distributed across 16 categories, grounded in real-world websites and supported by 918 design images and 1,255 individual test cases.
How Evaluation Works#
Rather than relying on a single automated metric, the researchers developed a two-part verification system. A graphical interface agent tests whether the generated code runs correctly, while a vision language model serves as a secondary judge to assess whether the output matches the intended design. This dual-component approach enables more nuanced measurement than traditional automated testing alone.
Key Findings#
Evaluations of leading visual language models across different coding-agent architectures revealed significant performance gaps at every difficulty tier. Most notably, even the strongest models currently available show substantial limitations when tackling full-stack development scenarios, suggesting that end-to-end website creation remains a challenging frontier for AI systems.
The work addresses a gap in how AI coding capabilities are typically assessed, moving beyond narrow benchmarks toward evaluating genuine, practical software development workflows.
