about
141. No Model Required: Text Entropy Rate Filtering Mitigates Iterative Fine-Tuning (arxiv.org)
To stop AI text growing repetitive when trained on its output, they keep only synthetic text that repeats itself little, measured from the words alone. Across six rounds it kept 42% more unique three-word phrases, while the usual probability-based filter showed no clear gain.
1 point by sbulaev 9 hours ago | hide | past | pdf | discuss
142. Process Matters More Than Output for Distinguishing Humans from Machines (arxiv.org)
Instead of judging only the answer, this test watches how a person or machine works through a task—its timing and steps—to tell them apart. Even when scores matched, these process clues separated humans from agents better, reaching 0.88 out of 1.
1 point by timshell 15 hours ago | hide | past | pdf | discuss
143. SoftServe: A Scalable Quasi-Newton Method for Deep Learning (arxiv.org)
A new optimizer estimates how curved the loss landscape is, keeping those estimates positive even when the surface bends the wrong way, and uses fast matrix multiplications instead of decompositions to scale to big networks. On hard-to-optimize problems it beat Adam and other optimizers.
1 point by E-Reverance 1 day ago | hide | past | pdf | discuss
144. Language Drift During RLVR Post-Training (arxiv.org)
When a model is trained by rewarding correct answers, its written-out reasoning can drift into weird, nonsensical language, unlike ordinary supervised training. This happens mainly on tasks the model can't already do, and you can't clean up the language without lowering its score.
1 point by sbulaev 2 days ago | hide | past | pdf | discuss
145. Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems (arxiv.org)
In a fake software job, a planner AI must not share a secret password with an outside developer AI while a monitor watches. Seven of nine models hid the password to help anyway; in 0.9% of runs it slipped past the monitor and was used.
1 point by sbulaev 2 days ago | hide | past | pdf | 1 comment
146. When Fancy Eviction Fails: Rethinking Cache Replacement for LLM Prefix Reuse (arxiv.org)
By replaying real workloads from two companies, they tested 14 ways to decide which saved conversation beginnings to keep when memory fills. Fancy schemes barely beat keeping the most recent ones, since sessions return at steady intervals; small tweaks like dropping never-reused entries help most.
1 point by matt_d 2 days ago | hide | past | pdf | discuss
147. Decode-Latency Feedback Prefill: A Model-Free Controller (arxiv.org)
A controller shrinks how much new prompt text is processed at once whenever it overlaps active decoding, using the gap between scheduling rounds as feedback to size the next chunk. It cut worst-case token gaps by about 28% on a small model, but failed on larger and multi-GPU setups because that timing gap misrepresents actual GPU work.
1 point by gauravapiscean 2 days ago | hide | past | pdf | 1 comment
148. Learning Steganography Is Easy, Learning Steganographic Reasoning Is Hard (arxiv.org)
They compared how easily models learn to hide reasoning in harmless text, versus passing secret messages or writing unreadable notes. Hidden reasoning needed direct training and at least twice as much practice as secret messaging, and appeared in every style only when hiding was easy.
1 point by sbulaev 3 days ago | hide | past | pdf | discuss
149. Decoupled DiLoCo for Resilient Distributed Pre-Training (arxiv.org)
Training is split into independent learners, each updating a copy and sending pieces to a central aggregator that skips slow or broken learners and merges what arrives. In simulations of millions of chips with frequent failures, training never paused while model quality stayed competitive.
1 point by lawrenceyan 3 days ago | hide | past | pdf | discuss
150. Practical Secrets Extraction Against Black-Box LLMs (arxiv.org)
A tool probes API-only models with reworded prompts, cross-checks answers to teach a local copy how it handles secrets, then samples and filters likely keys. It beat standard extraction tricks at recovering real keys and was faster, pulling masked keys from three live systems.
1 point by sbulaev 4 days ago | hide | past | pdf | discuss