In plain words: Scanned 304 people listening to short stories to test whether the brain predicts words further ahead than the usual next-word training of language models. Adding long-range predictions matched brain activity better, with front-of-brain areas forecasting more distant, abstract ideas than side areas.
Abstract
Deep learning has recently made remarkable progress in natural language processing. Yet, the resulting algorithms remain far from competing with the language abilities of the human brain. Predictive coding theory offers a potential explanation to this discrepancy: while deep language algorithms are optimized to predict adjacent words, the human brain would be tuned to make long-range and hierarchical predictions. To test this hypothesis, we analyze the fMRI brain signals of 304 subjects each listening to 70min of short stories. After confirming that the activations of deep language algorithms linearly map onto those of the brain, we show that enhancing these models with long-range forecast representations improves their brain-mapping. The results further reveal a hierarchy of predictions in the brain, whereby the fronto-parietal cortices forecast more abstract and more distant representations than the temporal cortices. Overall, this study strengthens predictive coding theory and suggests a critical role of long-range and hierarchical predictions in natural language processing.
Charlotte Caucheteux, Alexandre Gramfort, Jean-Remi King
arXiv:2111.14232 · q-bio.NC, cs.AI, cs.CL, cs.LG, cs.NE · submitted Nov 28, 2021
abstract · pdf · html