about
Framework for identification of chaotic systems with symbolic regression (arxiv.org)
32 points by PaulHoule on Jan 2, 2024 | hide | past | pdf | 9 comments on HN

In plain words: A small neural network learns how a chaotic system changes over time from sparse, noisy measurements, then a search tool turns those learned pieces into explicit equations. It recovered the correct formulas and missing terms for three known chaotic systems, matching their true math.

Abstract · AI-Lorenz: A physics-data-driven framework for black-box and gray-box identification of chaotic systems with symbolic regression

Discovering mathematical models that characterize the observed behavior of dynamical systems remains a major challenge, especially for systems in a chaotic regime. The challenge is even greater when the physics underlying such systems is not yet understood, and scientific inquiry must solely rely on empirical data. Driven by the need to fill this gap, we develop a framework that learns mathematical expressions modeling complex dynamical behaviors by identifying differential equations from noisy and sparse observable data. We train a small neural network to learn the dynamics of a system, its rate of change in time, and missing model terms, which are used as input for a symbolic regression algorithm to autonomously distill the explicit mathematical terms. This, in turn, enables us to predict the future evolution of the dynamical behavior. The performance of this framework is validated by recovering the right-hand sides and unknown terms of certain complex, chaotic systems such as the well-known Lorenz system, a six-dimensional hyperchaotic system, and the non-autonomous Sprott chaotic system, and comparing them with their known analytical expressions.

Mario De Florio, Ioannis G. Kevrekidis, George Em Karniadakis
arXiv:2312.14237 · physics.comp-ph, cs.LG, nlin.CD, physics.data-an · submitted Dec 21, 2023
abstract · pdf · html · 28 pages, 15 figures, 9 tables

add comment on HN

Admittedly I've only had time to skim the paper once, but I'm having some trouble understanding what the neural network is doing here--is it determining the order of the system (the number of first-order ODEs we need to represent it)?
It’s actually finding the ODE, which is exactly what symbolic regression is about.
The (perhaps naive) approach I've taken in the past is to search the space of ODE systems using genetic programming with a loss function obtained by integrating my candidate ODE system and measuring how closely the output matches the data I want to perform regression on.

What they're doing here is not that. I think if I understand it correctly they're using the neural network to generate data which when symbolically regressed with PySR yield the RHS of each ODE in the system.

What's not immediately obvious is what the benefit of introducing the neural network is--does it make it faster than the "direct" naive approach?

not sure on details but i think neural networks have pretty great properties for finding center manifolds
My first thought was 'let's try it on resting-state fmri data'.
can you say more about the application there? what does the data look like and why is symbolic regression important? asking for a friend ;).
well it was my first thought, and I haven't had my coffee yet.

rsfmri data comprise multiple timeseries sampled from voxels throughtout the brain. The dynamics are complex, and there have been attempts to examine it through the lens of attractors etc. I'm not an expert in chaotic analysis, but will say that most of the advances in the field of neuroscience come from innovative analytic methods. Like I said, it was my first thought, but there is a whole subfield examining brain dynamics through these lenses.

That kind of chaos is about simple systems that are unpredictable in the long term. That Lorenz system is just a few equations you could write on a napkin. Whereas your brain has billions and billions of neurons. (You might blame the complexity on having a lot of parts.)

That said, another aspect of dissipative chaos is that a highly complex system like turbulence (10^28 or so atoms bouncing off each other) often can be described by an attractor that is rather low dimensional and could be modeled with a few equations. So you might have some neural signal that tools like that could make an interesting model of.

as long as the dynamics are relatively low dimensional, it should be possible to estimate the differential equation, even if it is observed in a high dimensional space