In plain words: Three leading image-and-text AI systems took 51 standard tests of vision used on humans, scored against healthy adults. They named objects well but failed basic skills like judging orientation, position, and occlusion at levels that would signal a visual problem in a person.
Abstract · Visual Language Models show widespread visual deficits on neuropsychological tests
Visual Language Models (VLMs) show remarkable performance in visual reasoning tasks, successfully tackling college-level challenges that require high-level understanding of images. However, some recent reports of VLMs struggling to reason about elemental visual concepts like orientation, position, continuity, and occlusion suggest a potential gulf between human and VLM vision. Here we use the toolkit of neuropsychology to systematically assess the capabilities of three state-of-the-art VLMs across visual domains. Using 51 tests drawn from six clinical and experimental batteries, we characterise the visual abilities of leading VLMs relative to normative performance in healthy adults. While the models excel in straightforward object recognition tasks, we find widespread deficits in low- and mid-level visual abilities that would be considered clinically significant in humans. These selective deficits, profiled through validated test batteries, suggest that an artificial system can achieve complex object recognition without developing foundational visual concepts that in humans require no explicit training.
Gene Tangtartharakul, Katherine R. Storrs
arXiv:2504.10786 · cs.CV, cs.AI, cs.LG · submitted Apr 15, 2025 · updated Apr 16, 2025
abstract · pdf · 31 pages, 3 figures, 1 supplementary document with 1 figure and 51 sample images; corrected typo in Fig 1