In plain words: Two kinds of AI-text detectors—ones that score text without training and ones trained on examples—were tested on new topics, new writing tools, and lightly rewritten text. Both broke down outside their training conditions: trained ones dropped sharply, untrained ones hinged on the reference text.
Abstract
The rapid adoption of LLMs has increased the need for reliable AI text detection, yet existing detectors often fail outside controlled benchmarks. We systematically evaluate 2 dominant paradigms (training-free and supervised) and show that both are brittle under distribution shift, unseen generators, and simple stylistic perturbations. To address these limitations, we propose a supervised contrastive learning (SCL) framework that learns discriminative style embeddings. Experiments show that while supervised detectors excel in-domain, they degrade sharply out-of-domain, and training-free methods remain highly sensitive to proxy choice. Overall, our results expose fundamental challenges in building domain-agnostic detectors. Our code is available at: https://github.com/HARSHITJAIS14/DetectAI
Jivnesh Sandhan, Harshit Jaiswal, Fei Cheng, Yugo Murawaki
arXiv:2601.15301 · cs.CL, cs.AI · submitted Jan 9, 2026 · updated Jan 27, 2026
abstract · pdf · html