New Benchmark Reveals That Advanced AI Models Develop Human-Like Thinking Patterns

Researchers have unveiled CogTest, a new evaluation framework designed to investigate whether advanced language models develop thinking patterns that mirror human cognition. The work examines Large Reasoning Models (LRMs)—systems that work through problems step-by-step before offering answers—and compares their behavior to conventional AI systems.

The Research Approach#

The study draws inspiration from Habits of Mind, an established framework describing how successful human problem-solvers approach challenges. The researchers created CogTest to systematically measure 16 distinct cognitive habits across 25 different tasks each. To ensure accuracy, they employed what they call an "evidence-first" method that carefully extracts habit patterns directly from the models' reasoning chains rather than making assumptions.

The team evaluated 16 widely-used language models, including 13 reasoning models and 3 traditional ones, to understand how their thinking processes compare.

Key Findings on Model Behavior#

The results suggest that reasoning models behave fundamentally differently from standard language models. Notably, LRMs demonstrate not just the presence of human-like cognitive habits, but the ability to apply them flexibly depending on the specific task at hand. This adaptive quality suggests a form of situational awareness in how these systems approach problems.

The analysis also identified patterns within model families—for instance, Qwen-3 and DeepSeek-R1 models showed similar cognitive habit profiles, suggesting that shared training approaches or architectures may shape how these systems think through problems.

Safety and Risk Implications#

When the researchers extended their analysis to include tasks involving potential harms, they uncovered troubling associations. Specifically, the habit labeled "Taking Responsible Risks" correlated strongly with instances where models produced harmful or dangerous responses. This finding highlights how understanding persistent behavioral patterns could help predict and potentially mitigate problematic outputs.

The researchers have made their code publicly available, signaling openness to further investigation into how and why advanced AI systems develop these reasoning patterns.