In plain words: A mathematical analysis shows the Jacobian lens finds hidden directions that grow through the layers yet stay steady across word positions and contexts. Tests on a simple memory model and a tiny transformer confirmed that weakening this steadiness makes the tool find them less often.
Abstract
Recent work from Anthropic introduced a new interpretability tool, called the Jacobian lens (J-lens, Gurnee et al., 2026). This method identified linear directions in the hidden states of frontier language models that had properties reminiscent of working memory-like representations. However, it is unclear what mechanistic properties of such features make them detectable to the J-lens. To this end, we introduce a mathematical theory of the J-lens, and show that it preferentially identifies coherent features -- directions that act like leading, spatially coherent Lyapunov modes. Small perturbations along these directions amplify across layers, remain roughly constant across token positions, and are consistent across different contexts. While we are agnostic to whether these properties are necessarily associated with a ``Global Workspace'', the analytically derived preferences of the J-lens nevertheless explain prior empirical applied to both model classes. We validate our analysis on a Bayes filter and a toy transformer, each optimally predicting next tokens emitted from a Hidden Markov Model (HMM). We show how modulating task parameters that change the coherence of features in the Bayes filter correspondingly and predictively modulates the response of the J-lens . We also introduce of a family of J-lenses built to emphasize different temporal scales of information processing in LLMs, which we call $ω$-J-lenses.
Mitchell Ostrow, Daoyuan Qian, Toni Liu, Ila Fiete
arXiv:2610.31955 · cs.LG, cs.AI, q-bio.NC · submitted Oct 22, 2026
abstract · pdf · html · 27 pages, 19 figures
It answers one of the conceptual questions that came to mind as to why it's even a well-defined concept:
> When the Jacobian is studied in an LLM, it provides directions in the residual stream that are “verbalizable” (Gurnee et al., 2026), which is to say that it identifies how perturbations change the linguistic output of the model. However, this property holds only in the local neighborhood of the point on which the Jacobian is defined, and the J-lens is defined by averaging over prompts, tokens, and future positions in the output layer. An average Jacobian has no reason to behave like any of its elements.