Vision Language Models Fabricate Illusions in Normal Images

Perceptual illusions have long served as windows into how humans and animals process visual information—revealing the gap between what truly exists in an image and what our brains perceive. Now researchers are turning this tool inside out to diagnose weaknesses in artificial vision systems.

In a study submitted to arXiv, Tomer Ullman explores a novel failure mode in current vision language models: these AI systems see illusions where none exist. Rather than asking whether models fall prey to the same illusions humans do, the research presents them with "illusion-illusions"—images that resemble illusions but actually depict straightforward, unambiguous scenes.

What Are Illusion-Illusions?#

The images in question include ordinary scenarios: a standard duck, lines that genuinely are crooked (not merely appearing so), circles of different sizes that truly differ in diameter. These are not perceptual tricks; they depict reality as it is. Yet many current vision language models misidentify them as illusions, claiming to perceive distortions or ambiguities that aren't present.

A Symptom of Deeper Issues#

Ullman frames these failures as part of a broader pattern of processing errors already documented in the literature on AI visual understanding. The misidentification of non-illusions as illusions suggests fundamental gaps in how these models encode and reason about visual information—gaps that may contribute to other known limitations in their perceptual and reasoning capabilities.

Why It Matters#

The finding underscores that vision language models lack the robust visual grounding that humans develop through embodied experience. By examining where and how these systems go wrong, researchers gain insights into the architectural and training-related weaknesses that need addressing as these models become more integral to real-world applications.