In plain words: A model that breaks lines in fixed-width text counts characters by building curved internal maps, twisting them to judge distance to the edge, then making a yes-or-no call. Tests that altered these maps confirmed the trick and exposed visual illusions that fool the counting.
Abstract · When Models Manipulate Manifolds: The Geometry of a Counting Task
Language models can perceive visual properties of text despite receiving only sequences of tokens-we mechanistically investigate how Claude 3.5 Haiku accomplishes one such task: linebreaking in fixed-width text. We find that character counts are represented on low-dimensional curved manifolds discretized by sparse feature families, analogous to biological place cells. Accurate predictions emerge from a sequence of geometric transformations: token lengths are accumulated into character count manifolds, attention heads twist these manifolds to estimate distance to the line boundary, and the decision to break the line is enabled by arranging estimates orthogonally to create a linear decision boundary. We validate our findings through causal interventions and discover visual illusions--character sequences that hijack the counting mechanism. Our work demonstrates the rich sensory processing of early layers, the intricacy of attention algorithms, and the importance of combining feature-based and geometric views of interpretability.
Wes Gurnee, Emmanuel Ameisen, Isaac Kauvar, Julius Tarng, Adam Pearce, Chris Olah, Joshua Batson
arXiv:2601.04480 · cs.LG · submitted Jan 8, 2026
abstract · pdf · html