about
Towards a Science of Scaling Agent Systems (arxiv.org)
3 points by gpi 176 days ago | hide | past | pdf | discuss on HN

In plain words: Testing 260 single- and multi-agent setups, a new formula predicts performance from model skill, coordination, and task shape. It picked the best setup 87% of the time, showing teamwork helps on splittable tasks but hurts on step-by-step ones.

Abstract

Agents, language model-based systems capable of reasoning, planning, and acting are widely adopted in real-world tasks, yet how their performance changes as these systems scale across key dimensions remains underexplored. We introduce quantitative scaling principles for agent systems as a predictive model, capturing how performance varies with coordination, model capability, and measurable system and task factors. Across 260 configurations spanning six agentic benchmarks, five canonical architectures (Single-Agent and four Multi-Agent: Independent, Centralized, Decentralized, Hybrid), and three LLM families, we perform controlled evaluations, standardizing tools, prompts, and compute to isolate architectural effects. The resulting model achieves a cross-validated R^2=0.373 across all six benchmarks (R^2=0.413 with a task-grounded capability metric). We identify a robust capability-saturation effect and additional patterns: (1) a coordination yields diminishing returns once single-agent baselines exceed certain performance; (2) tool-heavy tasks appear to incur multi-agent overhead; and (3) architectures without centralized verification tend to propagate errors more than those with centralized coordination. Relative performance change compared to single-agent baseline ranges from +80.8% on decomposable financial reasoning to -70.0% on sequential planning, demonstrating that architecture-task alignment determines collaborative success. The framework identifies the best-performing architecture for 87% of held-out configurations and shows consistent relative architecture preferences on unseen frontier models. Agent effectiveness depends on alignment between coordination and task structure, and that mismatched coordination degrades the performance.

Yubin Kim, Ken Gu, Chanwoo Park, Chunjong Park, Samuel Schmidgall, A. Ali Heydari, Yao Yan, Zhihan Zhang, Yuchen Zhuang, Yun Liu, Mark Malhotra, Paul Pu Liang, et al.
arXiv:2512.08296 · cs.AI · submitted Dec 9, 2025 · updated Apr 8, 2026
abstract · pdf · html

add comment on HN
Also discussed: Mar 2026 (1 point, 0 comments) · Jan 2026 (1 point, 1 comment) · Dec 2025 (1 point, 0 comments) · Dec 2025 (1 point, 0 comments) · Dec 2025 (2 points, 0 comments)