What Makes LLMs Think Like Humans? Researchers Identify Key Computational Factors

Researchers have systematically investigated what computational choices help large language models develop conceptual representations that align with human thinking. The work sidesteps a fundamental problem in AI evaluation: most existing benchmarks don't actually measure how closely a model's internal representations match the way people organize concepts.

How the Study Measured Human Alignment#

To address this gap, the team evaluated 75+ language models using a triplet similarity task—a method borrowed from cognitive science where humans and models judge which two concepts are most alike. They drew test concepts from the THINGS database, ensuring the task reflected real conceptual relationships rather than benchmark artifacts.

This approach sidesteps the limitations of standard AI benchmarks, which can yield high scores without revealing whether a model actually represents concepts the way humans do.

What Matters Most (And What Doesn't)#

The findings challenge some common assumptions about LLM design:

• Instruction fine-tuning emerged as one of the strongest drivers of human-like representation
• Larger attention head dimensionality significantly improved alignment
• Parameter size had surprisingly limited effect
• Activation function choice and multimodal pretraining showed minimal influence on representational alignment

The results suggest that how models are trained and structured matters more than simply making them bigger or adding additional modalities.

A Gap in Current Evaluation#

The researchers also tested how well existing benchmarks—including BigBenchHard and MUSR—predicted human-model alignment. While some benchmarks correlated more strongly with representational alignment than others, none explained the full picture. This disconnect reveals a critical weakness: popular AI benchmarks may not be tracking progress toward human-like cognition as effectively as assumed.

The work highlights both a methodological path forward and a concrete set of design priorities for building LLMs that genuinely function as models of human thought.