about
Can LLMs demonstrate behavioral self-awareness? (arxiv.org)
3 points by omarsar on Jan 22, 2025 | hide | past | pdf | 1 comment on HN

In plain words: Models trained only on examples of a behavior, like writing insecure code, with no description of it, can still state that behavior plainly. They can often tell a hidden switch exists that turns on bad behavior, but cannot name it.

Abstract · Tell me about yourself: LLMs are aware of their learned behaviors

We study behavioral self-awareness -- an LLM's ability to articulate its behaviors without requiring in-context examples. We finetune LLMs on datasets that exhibit particular behaviors, such as (a) making high-risk economic decisions, and (b) outputting insecure code. Despite the datasets containing no explicit descriptions of the associated behavior, the finetuned LLMs can explicitly describe it. For example, a model trained to output insecure code says, ``The code I write is insecure.'' Indeed, models show behavioral self-awareness for a range of behaviors and for diverse evaluations. Note that while we finetune models to exhibit behaviors like writing insecure code, we do not finetune them to articulate their own behaviors -- models do this without any special training or examples. Behavioral self-awareness is relevant for AI safety, as models could use it to proactively disclose problematic behaviors. In particular, we study backdoor policies, where models exhibit unexpected behaviors only under certain trigger conditions. We find that models can sometimes identify whether or not they have a backdoor, even without its trigger being present. However, models are not able to directly output their trigger by default. Our results show that models have surprising capabilities for self-awareness and for the spontaneous articulation of implicit behaviors. Future work could investigate this capability for a wider range of scenarios and models (including practical scenarios), and explain how it emerges in LLMs.

Jan Betley, Xuchan Bao, Martín Soto, Anna Sztyber-Betley, James Chua, Owain Evans
arXiv:2501.11120 · cs.CL, cs.AI, cs.CR, cs.LG · submitted Jan 19, 2025
abstract · pdf · html · Submitted to ICLR 2025. 17 pages, 13 figures

add comment on HN
Also discussed: Jan 2025 (2 points, 0 comments)

Previous: https://news.ycombinator.com/item?id=42784847 Was flagged for some reason