I get more than a little annoyed with the advertising-like tone in this and several other articles around this general class of method/trick for speeding up complex simulations.

It's not that it's not a useful method (it is). It's that it misrepresents the utility of the general "build an exact simulation, then train a regressor on it to make fast approximations" approach. It's very useful in certain situations (repeated calculations on similar parameters) and completely useless in others.

The key issue is that most of the slow models you'd want to use this on are highly non-linear. In certain regions of the parameter space, very small changes in input result in very large changes in output. This is fine, so long as you know where all of these regions are and can capture them in your training data. That's easier for some problems than others. Even assuming you do know how/where to collect dense training data, this approach is only useful within the bounds of the training data you collect. It's relatively uncommon (but not super rare) that you want to repeatedly run a complex simulation within the same parameter space. This method is great when you do want/need to do that, and useless otherwise.

You have to understand that what you've trained is little more than a look up table. Anything that claims it can actually learn highly non-linear and irregular behavior well outside of the training dataset's bounds is snake oil.

Understand where this general class of technique is useful and where it isn't and ignore overblown claims. The entire abstract here is overblown hogwash. The actual paper is relatively interesting, but I really wish folks would drop the absurd advertising language and focus on what distinguishes this from the hundreds of very similar studies/methods on this topic.

Sorry for the rant... I just feel like abstracts are the most important part of a paper and get annoyed with poor ones. A better abstract would arguably be something they have a few paragraphs into their introduction:

"To construct high fidelity emulators with limited training data, the machine learning models need to have a good prior on the simulation models. Most work to date in building emulators using random forests, Gaussian Processes,or other machine learning models, do not fully capture the correlation among the output points,limiting their accuracy in emulating simulations with one, two, or three-dimensional output signals. On the other hand, convolutional neural network (CNN) have shown to have a good prior on natural signals, making them suitable for processing natural n-dimensional signals. However, as the CNN priors inherently rely on their architectures,8one has to find an architecture that gives the suitable prior of a given problem. Manually searching for the right architecture can take a significant amount of time and domain-specific expertise and often produces sub-optimal results.

Here we propose to solve this problem by employing efficient neural architecture search to simultaneously find the neural network architecture that is best suited for a given case and train it. With the efficient neural architecture search and a novel super-architecture presented in this work, the algorithm can find and train fast emulators for a wide range of applications while offering major improvements in terms of accuracy compared with other techniques, even when the training data is limited."

Without being cynical, I think the authors surely know this. Writing this way is exploiting the DL fad for career stepping stones. This is simply how science works these days: a lot of good science is done but those climbing latter boast about it or lose out.

Basically what jofer said...

I work in a field of condensed matter simulation, and the claim seems bizarre, empty, meaningless. Basically it's not that original, or difficult, to train a linearly-scaling with system size machine learning model on an underlying physics simulation that scales very slowly.

Don't get me wrong, it's still very cool and satisfying to get reasonably accurate results on my laptop in a few seconds what would take hours, or days, on supercomputer.

Hopefully this only ever gets used for exploration, not verification.

Why not?

https://en.wikipedia.org/wiki/Four_color_theorem

Could you explain the link you're making here? As far as I understand four colour theorem was computer assisted proof in the sense that mathematicians wrote a very specific and analytical definition for it in a logical theorem prover.

The linked article is very different right? Because it's using a neural net to do some kind of probabilistic inference as a heuristic for imperfectly stimulating physical events.

Exactly; it's about generating (much faster) heuristics for "guessing" the outcome given certain parameters, instead of running the real simulation every time. Useful, for sure, but the real thing should be used to verify once you've found something interesting.

My bad, I though it was a deterministic system, not probabilistic.