about
Moravec's Paradox: Towards an Auditory Turing Test (arxiv.org)
2 points by belter on Aug 4, 2025 | hide | past | pdf | discuss on HN

In plain words: A new test of 917 listening challenges—speech over speech, noise, distortions, and illusions—checks whether AI hears as well as people do. The best system answered only 6.9% correctly versus 52% for humans, with overall failure above 93%.

Abstract

This research work demonstrates that current AI systems fail catastrophically on auditory tasks that humans perform effortlessly. Drawing inspiration from Moravec's paradox (i.e., tasks simple for humans often prove difficult for machines, and vice versa), we introduce an auditory Turing test comprising 917 challenges across seven categories: overlapping speech, speech in noise, temporal distortion, spatial audio, coffee-shop noise, phone distortion, and perceptual illusions. Our evaluation of state-of-the-art audio models including GPT-4's audio capabilities and OpenAI's Whisper reveals a striking failure rate exceeding 93%, with even the best-performing model achieving only 6.9% accuracy on tasks that humans solved at 7.5 times higher success (52%). These results expose focusing failures in how AI systems process complex auditory scenes, particularly in selective attention, noise robustness, and contextual adaptation. Our benchmark not only quantifies the human-machine auditory gap but also provides insights into why these failures occur, suggesting that current architectures lack fundamental mechanisms for human-like auditory scene analysis. The traditional design of audio CAPTCHAs highlights common filters that humans evolved but machines fail to select in multimodal language models. This work establishes a diagnostic framework for measuring progress toward human-level machine listening and highlights the need for novel approaches integrating selective attention, physics-based audio understanding, and context-aware perception into multimodal AI systems.

David Noever, Forrest McKee
arXiv:2507.23091 · cs.AI, cs.SD, eess.AS · submitted Jul 30, 2025
abstract · pdf

add comment on HN