about
Do not think pink elephant (arxiv.org)
2 points by belter on Apr 24, 2024 | hide | past | pdf | discuss on HN

In plain words: Telling an image generator not to draw something can make it draw that thing, just like telling a person not to think of a pink elephant. Rewording the warning like therapy talk cut these forbidden images by up to 48.22%.

Abstract · Do not think about pink elephant!

Large Models (LMs) have heightened expectations for the potential of general AI as they are akin to human intelligence. This paper shows that recent large models such as Stable Diffusion and DALL-E3 also share the vulnerability of human intelligence, namely the "white bear phenomenon". We investigate the causes of the white bear phenomenon by analyzing their representation space. Based on this analysis, we propose a simple prompt-based attack method, which generates figures prohibited by the LM provider's policy. To counter these attacks, we introduce prompt-based defense strategies inspired by cognitive therapy techniques, successfully mitigating attacks by up to 48.22\%.

Kyomin Hwang, Suyoung Kim, JunHoo Lee, Nojun Kwak
arXiv:2404.15154 · cs.CL, cs.AI · submitted Apr 22, 2024 · updated Oct 31, 2024
abstract · pdf · html · This paper is accepted in CVPR 2024 Responsible Generative AI Workshop (ReGenAI)

add comment on HN