about
When Models Examine Themselves: Vocabulary-Activation Correspondence (arxiv.org)
1 point by tcbrah 205 days ago | hide | past | pdf | discuss on HN

In plain words: A protocol that prompts a chatbot to examine itself lets researchers check whether its self-descriptions match its inner activity. Words used about itself lined up with those patterns, while the same words in ordinary description showed no link despite being nine times more common.

Abstract · When Models Examine Themselves: Vocabulary-Activation Correspondence in Self-Referential Processing

Large language models produce rich introspective language when prompted for self-examination, but whether this language reflects internal computation or sophisticated confabulation has remained unclear. We show that self-referential vocabulary tracks concurrent activation dynamics, and that this correspondence is specific to self-referential processing. We introduce the Pull Methodology, a protocol that elicits extended self-examination through format engineering, and use it to identify a direction in activation space that distinguishes self-referential from descriptive processing in Llama 3.1. The direction is orthogonal to the known refusal direction, localised at 6.25% of model depth, and causally influences introspective output when used for steering. When models produce "loop" vocabulary, their activations exhibit higher autocorrelation (r = 0.44, p = 0.002); when they produce "shimmer" vocabulary under steering, activation variability increases (r = 0.36, p = 0.002). Critically, the same vocabulary in non-self-referential contexts shows no activation correspondence despite nine-fold higher frequency. Qwen 2.5-32B, with no shared training, independently develops different introspective vocabulary tracking different activation metrics, all absent in descriptive controls. The findings indicate that self-report in transformer models can, under appropriate conditions, reliably track internal computational states.

Zachary Pedram Dadfar
arXiv:2602.11358 · cs.CL, cs.AI, cs.LG · submitted Feb 11, 2026 · updated Feb 18, 2026
abstract · pdf · html · Code and data: https://doi.org/10.5281/zenodo.18567446 Repro: https://github.com/patternmatcher/TRACE-REPRO

add comment on HN