about
The Platonic Representation Hypothesis (arxiv.org)
2 points by Anon84 159 days ago | hide | past | pdf | 1 comment on HN

In plain words: As AI models grow larger and see more data, their ways of organizing information become more alike across image and text systems. The bigger they get, the more similarly they judge which data points are close, hinting at one shared picture of reality.

Abstract

We argue that representations in AI models, particularly deep networks, are converging. First, we survey many examples of convergence in the literature: over time and across multiple domains, the ways by which different neural networks represent data are becoming more aligned. Next, we demonstrate convergence across data modalities: as vision models and language models get larger, they measure distance between datapoints in a more and more alike way. We hypothesize that this convergence is driving toward a shared statistical model of reality, akin to Plato's concept of an ideal reality. We term such a representation the platonic representation and discuss several possible selective pressures toward it. Finally, we discuss the implications of these trends, their limitations, and counterexamples to our analysis.

Minyoung Huh, Brian Cheung, Tongzhou Wang, Phillip Isola
arXiv:2405.07987 · cs.LG, cs.AI, cs.CV, cs.NE · submitted May 13, 2024 · updated Jul 25, 2024
abstract · pdf · html · Equal contributions. Project: https://phillipi.github.io/prh/ Code: https://github.com/minyoungg/platonic-rep

add comment on HN
Also discussed: May 2024 (34 points, 6 comments)

to me it sounds less like platonism but more like that real-world data lives on a low dimensional manifold, and this manifold is reconstructed by sufficiently capable models from different perspectives.