Conor Houghton
 |
Contact conor.houghton@bristol.ac.uk |
The latest news!
(2024-11-12) New paper:
Modeling nonlinear oscillator networks using physics-informed hybrid reservoir computing.
Andrew Shannon, Conor Houghton, David Barton, Martin Homer
(Under review, 2024)
arXiv: 2411.05867
Modelling dynamical systems using differential equations is often, maybe even typically, difficult: we don't know all the terms in the equations, we don't all the parameters. The modern alternative is to give the problem to a neural network, maybe an RNN; however, this can be a disaster, the RNN will learn the training data perfectly but find some cunning way to draw bizarre conclusions about what the system would do in just slightly different circumstances to the ones in the data. The neural network also commits the twin crimes of ignore our existing intuition for the behaviour of the system and supplies no new qualitative insight into what's going on.
Here we consider a hybrid; a neural network coupled to an expert model: the system of differential equations we believe captures some important aspects of the behaviour. If we used a `proper' RNN with end-to-end training it would just take over and not leave anything for the differential equations. Instead we use a reservoir computer, a recurrent neural network with fixed connections: all that is learned in a final, linear, read-out layer. This follows the ideas described in:
Pathak, J. et al. Hybrid forecasting of chaotic processes: Using machine learning in conjunction with a knowledge-based model. Chaos 28, 041101 (2018)
Karniadakis, G. E. et al. Physics-informed machine learning. Nature Reviews Physics 3, 422-440 (2021).
We test this physics-informed hybrid reservoir computing approach on an oscillating system: these have lots of applications including the application we ultimately have in mind, EEG signals recorded from the brain. We use a system of Kuramoto oscillators, so the idea is that the hybrid system includes a module, the expert model, which includes the differential equations for the Kuramoto oscillators. However, the ground truth, the thing we are trying to model, has an extra non-linear coupling: this represents extra physics that the modeller doesn't know about. This is a stern test, the extra term changes the behaviour making the expert model useless on its own. The question is whether the reservoir can compensate?
The answer is a bit complicated: the hybrid is mostly better than the alternative of using just the reservoir or just the expert model and the hybrid is much less sensitive to the choice of metaparameters, but all three approaches struggle a bit with the task.
Our conclusion is that the hybrid systems are likely to be useful for control systems, but our interest is using the hybrid systems to better fit the expert system: the idea is that the expert system contains important stuff, the parameters, if fit, will tell us about the system we're modelling. For example, if we are modelling EEG, the couplings will tell us something important about neural circuits. However, if extra unknown nonlinear terms mean we can't fit the model, we can't use the data to understand the system. Using a hybrid might allow us to fit the data and find the parameters, couplings and the like, that answer the real world questions that we might have. That's what we're going to look at next.