Study challenges claims that large language models have passed the Turing Test

Researchers have put large language models to a strict test based on Alan Turing's foundational proposal for measuring machine intelligence—and concluded that recent headlines declaring these systems have "passed the Turing Test" are premature.

The Test and Its Origins#

The Turing Test, first described by Turing in 1950, proposes a practical benchmark: a machine demonstrates human-like intelligence if an evaluator cannot distinguish it from a human during a blind conversation. While the concept has become iconic in discussions of artificial intelligence, many modern claims of passing it have diverged significantly from Turing's original framework and instructions.

What the Research Found#

Scientists replicated Turing's three-player imitation game with a large language model, this time adhering closely to the original guidelines while applying scientific standards where Turing's instructions were ambiguous. In their computer-imitates-human scenario, only one out of all participants failed to identify the LLM correctly—suggesting the machines still fall well short of reliable deception. Notably, evaluators needed more than five minutes to complete the task, with a control benchmark (a man imitating a woman) taking even longer. This timeframe contrasts with some earlier positive results that may have been influenced by shorter evaluation windows.

Implications for AI Assessment#

The findings highlight how the Turing Test remains relevant for evaluating machine intelligence, even as its original rigor has been compromised in recent discourse. Distinguishing between carefully framed demonstrations and genuine human-level performance matters for both scientific credibility and practical applications in commercial and legal contexts. The researchers expect the test to play a central role in assessing AI capabilities for years ahead.