ArticlePublisher preview available

Jet Lag Recovery: Synchronization of Circadian Oscillators as a Mean Field Game

DOI:10.1007/s13235-019-00315-1
Authors:
Rene A. Carmona at Princeton University
Rene A. Carmona
Christy V. Graves
Christy V. Graves
Christy V. Graves
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Read publisher preview
Download citation
To read the full-text of this research, you can request a copy directly from the authors.

Abstract and Figures

The Suprachiasmatic Nucleus (SCN) is a region in the brain that is responsible for controlling circadian rhythms. The SCN contains on the order of 10^4 neuronal oscillators which have a preferred period slightly longer than 24 hours. The oscillators try to synchronize with each other as well as responding to external stimuli such as sunlight exposure. A mean field game model for these neuronal oscillators is formulated with two goals in mind: 1) to understand the long time behavior of the oscillators when an individual remains in the same time zone, and 2) to understand how the oscillators recover from jet lag when the individual has traveled across time zones. In particular, we would like to study the claim that jet lag is worse after traveling east than west. Finite difference schemes are used to find numerical approximations to the mean field game solutions. Numerical results are presented and conjectures are posed. The numerics suggest the time to recover from jet lag is about the same for east versus west trips, but the cost the oscillators accrue while recovering is larger for eastward trips.
Figures - available from: Dynamic Games and Applications
This content is subject to copyright. Terms and conditions apply.
  • A preview of this full-text is provided by Springer Nature.
  • Learn more
Preview content only
Content available from Dynamic Games and Applications
This content is subject to copyright. Terms and conditions apply.
Dynamic Games and Applications (2020) 10:79–99
https://doi.org/10.1007/s13235-019-00315-1
Jet Lag Recovery: Synchronization of Circadian Oscillators
as a Mean Field Game
René Carmona1·Christy V. Graves1
Published online: 25 May 2019
© Springer Science+Business Media, LLC, part of Springer Nature 2019
Abstract
A mean field game is proposed for the synchronization of oscillators facing conflicting
objectives. Our motivation is to offer an alternative to recent attempts to use dynamical
systems to illustrate some of the idiosyncrasies of jet lag recovery. Our analysis is driven
by two goals: (1) to understand the long time behavior of the oscillators when an individual
remains in the same time zone, and (2) to quantify the costs from jet lag recovery when
the individual has traveled across time zones. Finite difference schemes are used to find
numerical approximations to the mean field game solutions. They are benchmarked against
explicit solutions derived for a special case. Numerical results are presented and conjectures
are formulated. The numerics suggest that the cost the oscillators accrue while recovering is
larger for eastward travel which is consistent with the widely admitted wisdom that jet lag
is worse after traveling east than west.
Keywords Mean field game ·Mean field control ·Jet lag ·Synchronization ·Oscillators ·
Partial differential equations ·Explicit solutions ·Perturbation analysis ·Numerical results ·
Ergodic
1 Introduction
Circadian rhythm refers to the oscillatory behavior of certain biological processes occur-
ring with a period close to 24 h. Recent interest in these biological processes has spawned
from the Nobel Prize winning work of Hall, Rosbash, and Young, who discovered molecular
mechanisms for controlling the circadian rhythm in fruit flies [20]. Examples of circadian
rhythms in animals include sleep/wake patterns, eating schedules, bodily temperatures, hor-
mone production, and brain activity. These oscillations can be entrained to the 24 h cycle of
Partially supported by NSF #DMS-1716673, ARO #W911NF-17-1-0578, and the NSF GRFP.
BChristy V. Graves
cjvaughn@princeton.edu
René Carmona
rcarmona@princeton.edu
1ORFE & PACM, Princeton University, Princeton, NJ 08544, USA
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
... The analysis of the coupled Kuramoto oscillators through a mean field game formalism is first explored by [22,23] proving bifurcation from incoherence to coordination by a formal linearization and a spectral argument. [6] further develops this analysis in their application to a jet-lag recovery model. We follow these pioneering studies and analyze the Kuramoto model as a discounted infinite horizon stochastic game in the limit when the number of oscillators goes to infinity. ...
... With finitely many oscillators, we consider the following version of the model already introduced in [6,23]. We fix a large integer N and for i ∈ {1, . . . ...
... However, they cannot model synchronization with external drivers, thus requiring additional terms. Indeed, the jet-lag recovery model of [6] introduce a cost for misalignment with the exogenously given sunlight frequency, providing an incentive to be in synch with the environment as well. These studies are clear evidences of the modeling potential of the mean field game formalism in all models when self organization is the salient feature. ...
Synchronization in a Kuramoto Mean Field Game
Preprint
  • Oct 2022
The classical Kuramoto model is studied in the setting of an infinite horizon mean field game. The system is shown to exhibit both synchronization and phase transition. Incoherence below a critical value of the interaction parameter is demonstrated by the stability of the uniform distribution. Above this value, the game bifurcates and develops self-organizing time homogeneous Nash equilibria. As interactions become stronger, these stationary solutions become fully synchronized. Results are proved by an amalgam of techniques from nonlinear partial differential equations, viscosity solutions, stochastic optimal control and stochastic processes.
View
... In an ergodic control problem, the target dynamics are optimized so that a timeaveraged objective function is minimized or maximized. This concept is suited to many real-world problems, including harvesting of renewable resources [9], chemostat control [10], operation of dam-reservoir systems [11], investment for sustainable economic growth [12], welfare accumulation [13], inventory control [14], mechanical systems management [15], physiological dynamics [16], and hydrodynamic problems [17]. Modeling environmental and ecological dynamics in rivers with the ergodic control, however, has been paid much less attention so far. ...
... There are two objectives in this sub-section. The first objective is to verify optimality of the exact solution in Proposition 1 and its associated control (16). We then show that this control is indeed optimal and that Ĥ H = . ...
... We have Ĥ H = as the unique effective Hamiltonian of the HJB equation (11). Furthermore, (16) is an optimal control. ...
Analytical and numerical solutions to ergodic control problems arising in environmental management
Preprint
  • Dec 2020
Environmental management should be based on a long-run sustainable viewpoint. A stochastic control problem optimizing a long-run objective function is an ergodic control problem whose resolution can be achieved by solving an associated Hamilton-Jacobi-Bellman (HJB) equation having an effective Hamiltonian. However, this key topic has not been well-studied in the context of environmental management. Focusing on non-smooth sediment storage management as an engineering problem, we formulate a new ergodic control problem under discrete observations: a natural assumption of realistic observations. We give an analytical solution to a simple problem and verify optimality and uniqueness of the corresponding HJB equation. To manage HJB equations in more general cases, we propose a fast sweep method resorting to neither pseudo-time integration nor vanishing discount. The optimal policy and the effective Hamiltonian are then computed simultaneously. A robust control counterpart where the dynamics involve uncertainties is also considered based on a multiplier robust formalism that harmonizes with the stochastic control approach. The resulting HJB equation has a stronger non-linearity but is managed using the presented numerical method. An extended problem constraining control policy is analyzed as well.
View
... Ergodic control is a mathematical concept for adaptively and sustainably controlling stochastic dynamics in a long run. 1 In an ergodic control problem, the target dynamics are optimized so that a time-averaged objective function is minimized or maximized. This concept is suited to many real-world problems, including harvesting of renewable resources, 2 chemostat control, 3 operation of dam-reservoir systems, 4 investment for sustainable economic growth, 5 welfare accumulation, 6 inventory control, 7 mechanical systems management, 8 physiological dynamics, 9 and hydrodynamic problems. 10 Modeling environmental and ecological dynamics in rivers with the ergodic control, however, has been paid much less attention so far. ...
... Now, we derive an exact classical solution to the HJB equation (6). We set Φ 0 ð Þ ¼ 0 following (9). This treatment can mitigate the singularity at x ¼ 0. The HJB equation in the present case is ...
Analytical and numerical solutions to ergodic control problems arising in environmental management
Article
Full-text available
Environmental management optimizing a long‐run objective is an ergodic control problem whose resolution can be achieved by solving an associated non‐local Hamilton–Jacobi–Bellman (HJB) equation having an effective Hamiltonian. Focusing on sediment storage management as a modern engineering problem, we formulate, analyze, and compute a new ergodic control problem under discrete observations: a simple but non‐trivial mathematical problem. We give optimality and comparison results of the corresponding HJB equation having unique non‐smoothness and discontinuity. To numerically compute HJB equations, we propose a new fast‐sweep method resorting to neither pseudo‐time integration nor vanishing discount. The optimal policy and the effective Hamiltonian are then computed simultaneously. Convergence rate of numerical solutions is computationally analyzed. An advanced robust control counterpart where the dynamics involve uncertainties is also numerically considered.
View
... Emergence of oscillations in mean-field processes is a longstanding issue in the literature, that is not specific to neuroscience applications, as it reflects a common feature of self-organization in statistical physics (see e.g. [18,19,38,29]), biological models of synchrony [12,1] or neuroscience [36,35,25]. We mention here specifically [25] where global oscillations have been proven for interacting Hawkes processes with circular connectivity. ...
... We now turn to the similar treatment of i = N A + 1, . . . , N : for such i, using the global Lipschitz continuity of G (12) δ N (t) ≤ c 6 Proceeding in the same way as before, we obtain, for some constant C 6 (t) > 0 ...
Interacting Hawkes processes with multiplicative inhibition
Preprint
Full-text available
  • May 2021
In the present work, we introduce a general class of mean-field interacting nonlinear Hawkes processes modelling the reciprocal interactions between two neuronal populations, one excitatory and one inhibitory. The model incorporates two features: inhibition, which acts as a multiplicative factor onto the intensity of the excitatory population and additive retroaction from the excitatory neurons onto the inhibitory ones. We give first a detailed analysis of the well-posedness of this interacting system as well as its dynamics in large population. The second aim of the paper is to give a rigorous analysis of the longtime behavior of the mean-field limit process. We provide also numerical evidence that inhibition and retroaction may be responsible for the emergence of limit cycles in such system.
View
... A canonical example of an interacting particle system is the coupled oscillators model of Kuramoto (Kuramoto, 1975;Strogatz, 2000;Dörfler and Bullo, 2014). Extensions of the classical Kuramoto model to mean-field games appears in (Yin et al., 2011;Carmona and Graves, 2020) and to FPF is given in (Tilton et al., 2012). ...
A Survey of Feedback Particle Filter and related Controlled Interacting Particle Systems (CIPS)
Preprint
Full-text available
  • Jan 2023
In this survey, we describe controlled interacting particle systems (CIPS) to approximate the solution of the optimal filtering and the optimal control problems. Part I of the survey is focussed on the feedback particle filter (FPF) algorithm, its derivation based on optimal transportation theory, and its relationship to the ensemble Kalman filter (EnKF) and the conventional sequential importance sampling-resampling (SIR) particle filters. The central numerical problem of FPF -- to approximate the solution of the Poisson equation -- is described together with the main solution approaches. An analytical and numerical comparison with the SIR particle filter is given to illustrate the advantages of the CIPS approach. Part II of the survey is focussed on adapting these algorithms for the problem of reinforcement learning. The survey includes several remarks that describe extensions as well as open problems in this subject.
View
... Equivalently, mean field games must be regarded as the continuum limit of games with a large number of symmetric players, each of them having a small effect on the dynamics of the whole group. The applications of mean field games are numerous, and spread across many disciplines, including social science (congestion [3,21,28]), cyber attacks [14]), biology (flocking [30,31], jet lag [13]), and economics (systemic risk [12], production of exhaustible resources [15,25]), just to name a few. As explained in [9,10], solutions to mean field games can be characterized through a coupled system of two forward and backward stochastic differential equations of mean field type, like those we address in this note. ...
Cemracs 2017: numerical probabilistic approach to MFG
Article
Full-text available
  • Jan 2019
This project investigates numerical methods for solving fully coupled forward-backward stochastic differential equations (FBSDEs) of McKean-Vlasov type. Having numerical solvers for such mean field FBSDEs is of interest because of the potential application of these equations to optimization problems over a large population, say for instance mean field games (MFG) and optimal mean field control problems. Theory for this kind of problems has met with great success since the early works on mean field games by Lasry and Lions, see [29], and by Huang, Caines, and Malhamé, see [26]. Generally speaking, the purpose is to understand the continuum limit of optimizers or of equilibria (say in Nash sense) as the number of underlying players tends to infinity. When approached from the probabilistic viewpoint, solutions to these control problems (or games) can be described by coupled mean field FBSDEs, meaning that the coefficients depend upon the own marginal laws of the solution. In this note, we detail two methods for solving such FBSDEs which we implement and apply to five benchmark problems. The first method uses a tree structure to represent the pathwise laws of the solution, whereas the second method uses a grid discretization to represent the time marginal laws of the solutions. Both are based on a Picard scheme; importantly, we combine each of them with a generic continuation method that permits to extend the time horizon (or equivalently the coupling strength between the two equations) for which the Picard iteration converges.
View
Interacting Hawkes processes with multiplicative inhibition
Article
In the present work, we introduce a general class of mean-field interacting nonlinear Hawkes processes modelling the reciprocal interactions between two neuronal populations, one excitatory and one inhibitory. The model incorporates two features: inhibition, which acts as a multiplicative factor onto the intensity of the excitatory population and additive retroaction from the excitatory neurons onto the inhibitory ones. We give first a detailed analysis of the well-posedness of this interacting system as well as its dynamics in large population. The second aim of the paper is to give a rigorous analysis of the longtime behavior of the mean-field limit process. We provide also numerical evidence that inhibition and retroaction may be responsible for the emergence of limit cycles in such system.
View
View
Nonlinear elliptic systems and mean eld games
Article
Full-text available
In this paper we consider a class of quasilinear elliptic systems of PDEs which arise in the mean eld games theory of J-M Lasry and P-L. Lions. We provide a wide range of su cient conditions for existence of solutions to these systems: on one hand the Hamiltonians ( rst-order terms) need to be at most quadratic in the gradients, on the other they can even grow arbitrarily provided that they do not \oscillate extremely much in the space variable" (a conditions expressed rigorously by means of certain inequalities that involve the space derivatives of the Hamiltonians). We concentrate on periodic conditions, but the same techniques allow to handle Dirichlet, Neumann...boundary conditions, or even the evolutive counterpart of these equations.
View
Iterative strategies for solving linearized discrete mean field games systems
Article
Full-text available
  • Jun 2012
Mean fields games (MFG) describe the asymptotic behavior of stochastic differential games in which the number of players tends to +∞. Under suitable assumptions, they lead to a new kind of system of two partial differential equations: a forward Bellman equation coupled with a backward Fokker-Planck equation. In earlier articles, finite difference schemes preserving the structure of the system have been proposed and studied. They lead to large systems of nonlinear equations in finite dimension. A possible way of numerically solving the latter is to use inexact Newton methods: a Newton step consists of solving a linearized discrete MFG system. The forward-backward character of the MFG system makes it impossible to use time marching methods. In the present work, we propose three families of iterative strategies for solving the linearized discrete MFG systems, most of which involve suitable multigrid solvers or preconditioners.
View
Long time average of mean field games
Article
Full-text available
We consider a model of mean field games system defined on a time interval [0,T] and investigate its asymptotic behavior as the horizon T tends to infinity. We show that the system, rescaled in a suitable way, converges to a stationary ergodic mean field game. The convergence holds with exponential rate and relies on energy estimates and the Hamiltonian structure of the system.
View
View
Resynchronization of circadian oscillators and the east-west asymmetry of jet-lag
Article
  • Jul 2016
Cells in the brain's Suprachiasmatic Nucleus (SCN) are known to regulate circadian rhythms in mammals. We model synchronization of SCN cells using the forced Kuramoto model, which consists of a large population of coupled phase oscillators (modeling individual SCN cells) with heterogeneous intrinsic frequencies and external periodic forcing. Here, the periodic forcing models diurnally varying external inputs such as sunrise, sunset, and alarm clocks. We reduce the dimensionality of the system using the ansatz of Ott and Antonsen and then study the effect of a sudden change of clock phase to simulate cross-time-zone travel. We estimate model parameters from previous biological experiments. By examining the phase space dynamics of the model, we study the mechanism leading to the difference typically experienced in the severity of jet-lag resulting from eastward and westward travel.
View
Bertrand & Cournot Mean Field Games
Article
  • Jan 2014
We study how continuous time Bertrand and Cournot competitions, in which firms producing similar goods compete with one another by setting prices or quantities respectively, can be analyzed as continuum dynamic mean field games.Interactions are of mean field type in the sense that the demand faced by a producer is affected by the others through their average price or quantity. Motivated by energy or consumer goods markets, we consider the setting of a dynamic game with uncertain market demand, and under the constraint of finite supplies (or exhaustible resources). The continuum game is characterized by a coupled system of partial differential equations: a backward HJB PDE for the value function, and a forward Kolmogorov PDE for the density of players. Asymptotic approximation enables us to deduce certain qualitative features of the game in the limit of small competition.The equilibrium of the game is further studied using numerical solutions, which become very tractable by considering the tail distribution function instead of the density itself. This also allows us to consider Dirac delta distributions to use the continuum game to mimic finite N-player nonzero-sum differential games, the advantage being having to deal with two coupled PDEs instead of N. We find that, in accordance with the two-player game, a large degree of competitive interaction causes firms to slow down production.Consumers benefit from market competition in the sense that high product substitutability drives average price down. The continuum system can therefore be used as an effective approximation to even small player dynamic games.
View
Bertrand and Cournot Mean Field Games
Article
  • Jun 2014
We study how continuous time Bertrand and Cournot competitions, in which firms producing similar goods compete with one another by setting prices or quantities respectively, can be analyzed as continuum dynamic mean field games.Interactions are of mean field type in the sense that the demand faced by a producer is affected by the others through their average price or quantity. Motivated by energy or consumer goods markets, we consider the setting of a dynamic game with uncertain market demand, and under the constraint of finite supplies (or exhaustible resources). The continuum game is characterized by a coupled system of partial differential equations: a backward HJB PDE for the value function, and a forward Kolmogorov PDE for the density of players. Asymptotic approximation enables us to deduce certain qualitative features of the game in the limit of small competition.The equilibrium of the game is further studied using numerical solutions, which become very tractable by considering the tail distribution function instead of the density itself. This also allows us to consider Dirac delta distributions to use the continuum game to mimic finite N-player nonzero-sum differential games, the advantage being having to deal with two coupled PDEs instead of N. We find that, in accordance with the two-player game, a large degree of competitive interaction causes firms to slow down production.Consumers benefit from market competition in the sense that high product substitutability drives average price down. The continuum system can therefore be used as an effective approximation to even small player dynamic games.
View
Long Time Average of Mean Field Games with a Nonlocal Coupling
Article
  • Jan 2013
We study the long time average, as the time horizon tends to infinity, of the solution of a mean field game system with a nonlocal coupling. We show an exponential convergence to the solution of the associated stationary ergodic mean field game. Proofs rely on semiconcavity estimates and smoothing properties of the linearized system.
View
On regularity of transition probabilities and invariant measures of singular diffusions under minimal conditions
Article
  • Nov 2001
Let A=(a ij ) be a matrix-valued Borel mapping on a domain Ω⊂ℝ d , let b=(b i ) be a vector field on Ω, and let L A,b φ=a ij ∂ x i ∂ x j φ+b i ∂ x i φ. We study Borel measures μ on Ω that satisfy the elliptic equation L A,b * μ=0 in the weak sense: ∫L A,b φdμ=0 for all φ∈C 0 ∞ (Ω). We prove that, under mild conditions, μ has a density. If A is locally uniformly nondegenerate, A∈H loc p,1 and b∈L loc p for some p>d, then this density belongs to H loc p,1 . Actually, we prove Sobolev regularity for solutions of certain generalized nonlinear elliptic inequalities. Analogous results are obtained in the parabolic case. These results are applied to transition probabilities and invariant measures of diffusion processes.
View
Finite Difference Methods for Mean Field Games
Article
Mean field type models describing the limiting behavior of stochastic differential game problems as the number of players tends to +∞ have been recently introduced by J.-M. Lasry and P.-L. Lions [C. R., Math., Acad. Sci. Paris 343, No. 9, 619–625 (2006; Zbl 1153.91009); ibid. 343, No. 10, 679–684 (2006; Zbl 1153.91010); Jpn. J. Math. (3) 2, No. 1, 229–260 (2007; Zbl 1156.91321)]. They may lead to systems of evolutive partial differential equations coupling a forward Bellman equation and a backward Fokker-Planck equation. The forward-backward structure is an important feature of this system, which makes it necessary to design new strategies for mathematical analysis and numerical approximation. In this survey, several aspects of a finite difference method used to approximate the previously mentioned system of partial differential equations are discussed, including: existence and uniqueness properties, a priori bounds on the solutions of the discrete schemes, convergence, and algorithms for solving the resulting nonlinear systems of equations. Some numerical experiments are presented. Finally, the optimal planning problem is considered, i.e., the problem in which the positions of a very large number of identical rational agents, with a common value function, evolve from a given initial spatial density to a desired target density at the final horizon time.
View