We show how combining these methods enables a Bayesian approach to infer the parameters of chaotic high-dimensional models and quantify their uncertainties in situations previously discussed as intractable ( Rougier, 2013). We couple CIL with local approximation MCMC (LA-MCMC) ( Conrad et al., 2016), which is a surrogate modeling technique that makes asymptotically exact posterior characterization feasible for computationally expensive models. Chaoticity is tamed by usingĬorrelation integral likelihood (CIL) ( Haario et al., 2015), which is able to constrain the parameters of chaotic dynamical systems.
The present works combines two recent methods to tackle these problems due to model chaoticity and computational cost. Since successful application of MCMC generally requires large numbers of model evaluations, performing Bayesian inference with MCMC is often not possible. Furthermore, modeling dynamical systems is often computationally very demanding, which makes sample generation time consuming.
This straightforward strategy – for instance, using the squared Euclidean distance between model outputs and data to construct a Gaussian likelihood – is, however, inadequate for chaotic models, where small changes in parameters, or even in the tolerances used for numerical solvers, can lead to arbitrarily large differences in model outputs ( Rougier, 2013). In practice, Bayesian inference is often realized via Markov chain Monte Carlo (MCMC) methods ( Gamerman, 1997 Robert and Casella, 2004). In fully Bayesian inference, the problem is further regularized by prescribing a prior distribution on the model state. In the Bayesian setting ( Gelman et al., 2013), modeling this mismatch probabilistically yields a likelihood function, which enables maximum likelihood estimation or fully Bayesian inference. Parameters of a dynamical system model are most commonly inferred by minimizing a cost function that captures model–observation mismatch ( Tarantola, 2005). This paper focuses on Bayesian approaches to parameter inference in settings where (a) model dynamics are chaotic, and (b) sequential observations of the system are obtained so rarely that the model behavior has become unpredictable. For this reason, parameter estimation in chaotic models is an important problem for a range of geophysical applications. For example, Monte Carlo methods may be used to simulate future climate variability, but the distribution of possible climates will depend on the parameters of the climate model, and using the wrong model parameter distribution will result in potentially biased results with inaccurate uncertainties. Commonly used examples of chaotic systems include climate, weather, and the solar system.Ī system being chaotic does not mean that it is random: the dynamics of models of chaotic systems are still determined by parameters, which may be either deterministic or random ( Gelman et al., 2013). Chaoticity means that state of a system sufficiently far in the future cannot be predicted even if we know the dynamics and the initial conditions very precisely. Time evolution of many geophysical dynamical systems is chaotic. We investigate the behavior of the resulting algorithm with two smaller-scale problems and then use a quasi-geostrophic model to demonstrate its large-scale application. Thus, we developĪn inexpensive surrogate for the log likelihood with the local approximation Markov chain Monte Carlo method, which in our simulations reduces the time required for accurate inference by orders of magnitude. This statistic is computationally expensive to simulate, compounding the usual challenges of Bayesian computation with physical models. Specifically, we construct a likelihood function suited to chaotic models by evaluating a distribution over distances between points in the phase space this distribution defines a summary statistic that depends on the geometry of the attractor, rather than on pointwise matching of trajectories. (i) measuring model–data mismatch by comparing chaotic attractors and (ii) mitigating the computational cost of inference by using surrogate models. We perform Bayesian inference of parameters in high-dimensional and computationally demanding chaotic dynamical systems by combining two approaches: Obvious remedies, such as averaging over temporal and spatial data to characterize the mean behavior, do not capture the subtleties of the underlying dynamics. These models cannot be calibrated with standard least squares or filtering methods if observations are temporally sparse. Estimating parameters of chaotic geophysical models is challenging due to their inherent unpredictability.