AI
New Knowledge Graph Benchmarks the Teaching Capabilities of AI Models in K-12 Education
Researchers released K12-KGraph, a curriculum-aligned dataset drawn from official Chinese textbooks to test how well large language models understand educational content structure and visual learning materials.
Researchers have introduced K12-KGraph, a new structured dataset and benchmark designed to measure how effectively large language models can navigate the way educational content is organized and presented in K-12 curricula. Rather than simply testing whether AI systems can answer exam questions, this work focuses on a deeper capability: understanding the relationships between concepts, prerequisite knowledge chains, and how material is visually grounded in textbooks.
What the Dataset Contains#
K12-KGraph brings together information from official People's Education Press textbooks spanning mathematics, physics, chemistry, and biology at the primary, middle, and high school levels. The knowledge graph contains nine distinct node types and fourteen types of relationships that capture both the logical structure of the curriculum and connections to visual elements such as diagrams and illustrations. The team also created K12-Train, a supervised learning corpus of 7,335 examples—combining 2,267 text-only question-answer pairs with 5,068 multimodal questions that incorporate images.
Benchmark Performance and Results#
The researchers built K12-Bench, a 23,640-question benchmark split into five task families: Ground (visual grounding), Prereq (prerequisite chains), Neighbor (related concepts), Evidence (supporting information), and Locate (finding material in the curriculum structure). Testing major models revealed significant gaps: Gemini-3-Flash achieved only 57 percent exact match accuracy, while Gemma-4-31B-IT reached 46 percent. The prerequisite and concept-neighbor tasks proved to be the most difficult for these systems.
Training and Optimization Outcomes#
When fine-tuned on domain-specific K12-Train data, models showed measurable improvement. Using the same budget (2,300 samples), the text-only K12-Train variant outperformed equally sized subsets from eight other mainstream instruction-tuning corpora when evaluated on broader benchmarks like GaokaoBench and EduEval. For vision-language models, the full K12-Train corpus delivered the strongest results across multiple evaluation sets (Gaokao-MM, MDK12-medium, and K12Vista), despite using fewer samples than competing baseline datasets, and demonstrated that combining textual and visual supervision improves performance beyond either modality alone.
