about
Long-range and hierarchical language predictions in brains and algorithms (arxiv.org)
19 points by alexwg on Dec 2, 2021 | hide | past | pdf | 2 comments on HN

In plain words: Scanned 304 people listening to short stories to test whether the brain predicts words further ahead than the usual next-word training of language models. Adding long-range predictions matched brain activity better, with front-of-brain areas forecasting more distant, abstract ideas than side areas.

Abstract

Deep learning has recently made remarkable progress in natural language processing. Yet, the resulting algorithms remain far from competing with the language abilities of the human brain. Predictive coding theory offers a potential explanation to this discrepancy: while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions. To test this hypothesis, we analyze the fMRI brain signals of 304 subjects each listening to 70min of short stories. After confirming that the activations of deep language algorithms linearly map onto those of the brain, we show that enhancing these models with long-range forecast representations improves their brain-mapping. The results further reveal a hierarchy of predictions in the brain, whereby the fronto-parietal cortices forecast more abstract and more distant representations than the temporal cortices. Overall, this study strengthens predictive coding theory and suggests a critical role of long-range and hierarchical predictions in natural language processing.

Charlotte Caucheteux, Alexandre Gramfort, Jean-Remi King
arXiv:2111.14232 · q-bio.NC, cs.AI, cs.CL, cs.LG, cs.NE · submitted Nov 28, 2021
abstract · pdf · html

add comment on HN

I don't believe that fMRI has sufficient temporal or spatial resolution to support the claims in this paper.
A typical voxel contains a million or so neurons, and only shows a correlate of the energy consumption in it. Assuming there’s some kind of mapping between an artificial NN and voxels is quite a step. So yeah, highly speculative IMO.