New benchmark tests how well AI models turn charts into code

Researchers have unveiled Chart2Code, a new benchmark designed to evaluate how well large multimodal models (LMMs) can understand charts and generate corresponding code. The framework addresses a practical gap in model evaluation by grounding tasks in real-world user workflows rather than abstract metrics.

A Three-Level Hierarchy#

Chart2Code organizes its assessment tasks into three escalating difficulty tiers. Level 1 (Chart Reproduction) asks models to recreate a chart based on a reference image and user instructions. Level 2 (Chart Editing) requires more sophisticated modifications—such as switching chart types or inserting new elements—that demand deeper understanding of chart structure. Level 3 (Long-Table to Chart Generation) poses the most complex challenge: converting large, dense tables into accurate visualizations that faithfully reflect the source data and match user requirements.

This hierarchical design mirrors how practitioners actually work with charts, scaling from straightforward copying to complex transformation tasks.

Scale and Evaluation#

The benchmark spans 2,023 total tasks across 22 different chart types. Critically, Chart2Code employs dual evaluation metrics that capture both technical correctness and visual quality. Assessments measure whether generated code runs properly, and separately evaluate how faithfully the rendered chart matches the intended output—a distinction that single-metric approaches often miss.

Performance Findings#

Testing included 25 state-of-the-art models, covering both proprietary systems (including GPT-5) and the latest open-source alternatives such as Qwen2.5-VL, InternVL3/3.5, MiMo-VL, and Seed-1.6-VL. Results revealed substantial challenges: even the highest-performing model (GPT-5) achieved only 0.57 on code-quality evaluation and 0.22 on chart-quality metrics when tackling the editing tasks, illustrating that Chart2Code poses a genuinely difficult problem for current systems.

Impact and Access#

The researchers expect Chart2Code to catalyze progress in multimodal reasoning and push development toward more capable, general-purpose language models. Code and datasets are publicly available, enabling the research community to build on this foundation.