about
51. MLIRBench – A Benchmark and Dataset for Compiler and Hardware-Aware Reasoning (arxiv.org)
A test set of 1,768 paired snippets of compiler code, built by auto-editing programs and running them to confirm the result, asks models to say whether two versions behave the same. The best got 92.65% right but only 5.04/10 at predicting the code's behavior.
2 points by matt_d 4 days ago | hide | past | pdf | discuss
52. ShadowPrefill: Keeping Your Next Model Warm for Model Routing in Agentic Serving (arxiv.org)
When an agent switches models mid-session, the new model must re-read the conversation because each model keeps its own cache. This system pre-fills that context on the other model using idle space, so a switch finds it ready and cuts token cost up to 46%.
2 points by matt_d 4 days ago | hide | past | pdf | discuss
53. Remote Labor Index: Measuring AI Automation of Remote Work (arxiv.org)
A test made of real, paid projects across many kinds of jobs lets AI agents try to finish whole tasks from start to end. The best agent automated just 2.5% of the work, far below what smart-test scores suggest.
2 points by doener 5 days ago | hide | past | pdf | 1 comment
54. DIMWIT: Directly Invoking Models Without Intensive Tuning (arxiv.org)
Instead of writing a special rulebook for each job, this system asks a pretrained model to pick the next action from the current state, goal, and choices. A small model met response-time targets using within 10% of the copies that hand-written scaling rules allocate.
2 points by matt_d 6 days ago | hide | past | pdf | discuss
55. E-means: outperforming k-means (in one-tenth the epochs) (arxiv.org)
e-means updates cluster centers as a smoothed running average of small point batches, with a formula setting its smoothing and batch size so nothing needs tuning. It beats standard k-means's clustering error in a tenth of the passes, cutting GPU time up to 45 times.
2 points by Anon84 7 days ago | hide | past | pdf | discuss
56. Doc2Harness: Benchmarking how well AI builds agent harnesses (arxiv.org)
A test has a model build the scaffolding that lets an agent use tools and edit files, then runs it in isolation and checks requests, files, exit code, and output. The best model passed 53% of cases, failing where shared features are split across modules.
2 points by waximabbax 7 days ago | hide | past | pdf | discuss
57. AutoCompress: Automated Hardware-Guided Model Compression (arxiv.org)
This tool shrinks AI models by aiming at speed and memory limits on specific hardware, using measured serving costs to trim each layer rather than just cutting parameter counts. It beat the leading pruning tool by up to 6.57 percentage points on three chips.
2 points by gmays 7 days ago | hide | past | pdf | discuss
58. I Think, Therefore I Attack (arxiv.org)
Attackers feed reasoning agents fake "thinking" text that copies the model's own internal reasoning style, then test it in a simulated hacking environment. This trick succeeded on up to half of tasks, while the same request in a normal user voice never worked.
2 points by gok 8 days ago | hide | past | pdf | discuss
59. Triton for MTIA: Bridging the Programming Model Gaps for Custom AI Accelerators (arxiv.org)
A high-level kernel language for GPUs got a new compiler backend and a few chip-specific additions so it can write code for Meta's custom AI accelerator. Its kernels ran about as fast as hand-tuned C++ and now run in production across about 60 model types.
2 points by matt_d 8 days ago | hide | past | pdf | discuss
60. Paper: Nubank cuts AI agent update cycles from 3 weeks to under 5 days (arxiv.org)
Fake customers chat with a support agent and its pretend tool responses, so teams can test thousands of conversations before real customers see anything. These trial scores tracked real-world scores closely, and the chosen setup lifted self-service by 8.82 points in a live test.
2 points by rkovashikawa 12 days ago | hide | past | pdf | discuss