about
ECO: An LLM-Driven Efficient Code Optimizer for Warehouse Scale Computers (arxiv.org)
2 points by bone_tag 115 days ago | hide | past | pdf | discuss on HN

In plain words: A system finds slow code across Google's fleet, then has an AI rewrite it and checks the change with tests and review before it ships. It has landed over 6,400 changes, 99.5% causing no rollbacks, saving several hundred thousand CPU cores' worth of computing.

Abstract

This paper introduces ECO, a system that automatically modifies source code to improve performance at scale. ECO overcomes the localization problem by combining fleet-wide continuous profiling to identify performance-critical code with an embedding-based search to pinpoint specific optimization candidates, guided by a mined dictionary of performance anti-patterns. It overcomes the reliability problem through a multi-stage verification approach that uses automated testing, LLM-based self-review, and post-deployment monitoring to ensure changes are both correct and effective. Fully productionized and deployed within Google's hyperscale production fleet, ECO has successfully landed over 6,400 commits, changing more than 25,000 lines of production code. Incorrect changes are caught before they are submitted to production, and 99.5% of the submitted commits did not cause any rollbacks. These optimizations have resulted in savings equivalent to several hundred thousand normalized CPU cores, showing that ECO makes LLM-based optimization both practical at scale and highly impactful in real-world settings.

Hannah Lin, Martin Maas, Maximilian Roquemore, Arman Hasanzadeh, Fred Lewis, Yusuf Simonson, Ameya Shringi, Hongwen Dai, Patrick Musau, Tzu-Wei Yang, Amir Yazdanbakhsh, Deniz Altinbüken, et al.
arXiv:2503.15669 · cs.SE · submitted Mar 19, 2025 · updated Sep 17, 2026
abstract · pdf

add comment on HN