about
AlphaGo Moment for Model Architecture Discovery (arxiv.org)
38 points by Jimmc414 on Jul 26, 2025 | hide | past | pdf | 7 comments on HN

In plain words: A system that invents new neural network designs on its own: it proposes ideas, writes the code, trains and tests them, instead of only searching a space humans already defined. After 1,773 experiments it produced 106 designs that beat the best human-made ones.

Abstract

While AI systems demonstrate exponentially improving capabilities, the pace of AI research itself remains linearly bounded by human cognitive capacity, creating an increasingly severe development bottleneck. We present ASI-Arch, the first demonstration of Artificial Superintelligence for AI research (ASI4AI) in the critical domain of neural architecture discovery--a fully autonomous system that shatters this fundamental constraint by enabling AI to conduct its own architectural innovation. Moving beyond traditional Neural Architecture Search (NAS), which is fundamentally limited to exploring human-defined spaces, we introduce a paradigm shift from automated optimization to automated innovation. ASI-Arch can conduct end-to-end scientific research in the domain of architecture discovery, autonomously hypothesizing novel architectural concepts, implementing them as executable code, training and empirically validating their performance through rigorous experimentation and past experience. ASI-Arch conducted 1,773 autonomous experiments over 20,000 GPU hours, culminating in the discovery of 106 innovative, state-of-the-art (SOTA) linear attention architectures. Like AlphaGo's Move 37 that revealed unexpected strategic insights invisible to human players, our AI-discovered architectures demonstrate emergent design principles that systematically surpass human-designed baselines and illuminate previously unknown pathways for architectural innovation. Crucially, we establish the first empirical scaling law for scientific discovery itself--demonstrating that architectural breakthroughs can be scaled computationally, transforming research progress from a human-limited to a computation-scalable process. We provide comprehensive analysis of the emergent design patterns and autonomous research capabilities that enabled these breakthroughs, establishing a blueprint for self-accelerating AI systems.

Yixiu Liu, Yang Nan, Weixian Xu, Xiangkun Hu, Lyumanshan Ye, Zhen Qin, Pengfei Liu
arXiv:2507.18074 · cs.AI · submitted Jul 24, 2025
abstract · pdf · html

add comment on HN

This could be a very big paper if its claims are reproducible. Like approaching attention is all you need big.

They discovered 106 new state-of-the-art linear attention architectures through a fully autonomous AI research loop. The authors are making comparisons to AlphaGo’s move 37.

>This could be a very big paper if its claims are reproducible. Like approaching attention is all you need big.

If it were it would have the on of the worst title imaginable and include one of the worst abstracts for any such paper.

It seriously is a very big red flag that a paper is doing this much to talk about how important it is. It is also essentially pre-writing the headlines for journalists to use, making this whole thing, at least, look like it is a PR stunt.

The part that is in principle amenable to replication is where they throw a lot of stuff at the wall and see what sticks. The part where they hype their own work, on the other hand... as a rule of thumb, if this really were a breakthrough on the level of AlphaGo, they wouldn't have to make that comparison themselves, someone else would be impressed enough to do it for them.
Let's definitely wait for replication, but I am honestly not that surprised that it works. I am surprised it took so long for people to give it a real try. It's such an ideal scenario: every experiment is conducted inside the computer, so there is no need to gather data in the real world, which is the pain point for most experiments in science. The LLM is therefore free to try a lot of different combinations and learn in real time what works and what doesn't.
Interesting work. Not super familiar with neural architecture search, but how do they ensure they’re not overfitting to the test set? Seems like they’re evaluating each model on the test set, and using that to direct future evolution. I get that human teams will often do the same, but wouldn’t the overfitting issues be magnified a lot by doing thousands of iterations of this?
Interesting paper - it will fascinating to see if it pans out.

The one thing I didn't see that would be good is some validation that the architecture(s) that perform best on large models are the same architectures that perform best on small models.

Ie validation the assumption that you can use small models with sma amounts of training/compute to determine the best architecture for large models and high training budgets.

Even if it doesn't translate it would still be very cool to be able to qui kly evolve better small models (1M to 400M params), but I believe the implied goal (and what everyone wants) is that this exploration and discovery of novel architectures would be applicable for the really big models as well.

If you could only ai discover larger models by spending OpenAi/Anthropic/... budgets per exploration then we're not really gaining much in terms of novel ideas as the cost (time and budget) would be too prohibitive.