New benchmark tests multimodal AI on humanities and social sciences reasoning

Researchers have introduced HSSBench, a specialized evaluation framework designed to assess how well multimodal large language models (MLLMs) handle tasks in humanities and social sciences—a domain long neglected by existing benchmarks that tend to focus on STEM subjects and general knowledge.

The gap in current benchmarks#

Most performance tests for MLLMs emphasize vertical, step-by-step reasoning patterns typical of mathematics and science. They overlook the horizontal, cross-disciplinary thinking demanded by humanities and social sciences work, where success requires linking abstract concepts to visual information and synthesizing knowledge across multiple connected fields. This oversight masks significant weaknesses in how current models handle complexity in these domains.

HSSBench's scope and design#

The new benchmark encompasses over 13,000 carefully curated samples across six core categories and supports evaluation in all six official United Nations languages. Rather than relying solely on automated data creation, the researchers developed a hybrid pipeline in which domain experts and AI agents collaborate to generate, refine, and validate each question. This human-in-the-loop approach aims to ensure that tasks genuinely capture the nuanced reasoning these fields demand.

Performance findings#

Testing more than 20 mainstream MLLMs revealed that even state-of-the-art models struggle substantially with HSSBench tasks. The results suggest that current AI systems lack robust capacity to internalize and connect knowledge across disciplinary boundaries—a critical skill for humanities and social science work.

Why it matters#

By highlighting these gaps, HSSBench is intended to drive development of models with stronger cross-disciplinary reasoning capabilities, pushing the field beyond narrow technical benchmarks toward more comprehensive, humanistic evaluation standards.