ArticlePDF Available

# A Free Energy Principle for Biological Systems

Authors:

## Abstract and Figures

This paper describes a free energy principle that tries to explain the ability of biological systems to resist a natural tendency to disorder. It appeals to circular causality of the sort found in synergetic formulations of self-organization (e.g., the slaving principle) and models of coupled dynamical systems, using nonlinear Fokker Planck equations. Here, circular causality is induced by separating the states of a random dynamical system into external and internal states, where external states are subject to random fluctuations and internal states are not. This reduces the problem to finding some (deterministic) dynamics of the internal states that ensure the system visits a limited number of external states; in other words, the measure of its (random) attracting set, or the Shannon entropy of the external states is small. We motivate a solution using a principle of least action based on variational free energy (from statistical physics) and establish the conditions under which it is formally equivalent to the information bottleneck method. This approach has proved useful in understanding the functional architecture of the brain. The generality of variational free energy minimisation and corresponding information theoretic formulations may speak to interesting applications beyond the neurosciences; e.g., in molecular or evolutionary biology.
Content may be subject to copyright.
A preview of the PDF is not available
... One construal casts active inference as a principled formulation of the Bayesian brain hypothesis (Knill & Pouget, 2004) equipped with motor reflexes (Friston, 2010). A complementary construal begins by asking how organisms maintain separation from their environment during their lifespan (Friston, 2012). ...
... Beginning from this second construal, start with a graph (a set of nodes and edges) that is separated into internal states (children's neurocognitive states) and external states (the child-external setting). Separation invokes a statistical construct called a Markov blanket (Pearl, 1988;Friston, 2012). Markov blankets separate internal from external states by inducing a set of conditional independencies between them, called blanket states (Figure 2). ...
... Maintenance of the phenotype requires action that counters unexpected fluctuations (Bruineberg et al., 2018). The Markov blanket renders this an inference problem, in which internal state dynamics entail a generative model of hidden states (Friston, 2012). Generative models encode Bayesian beliefs about how hidden states cause sensation. ...
Preprint
Full-text available
This article reviews experimental evidence for the dependence of 2- to 5-year-olds’ linguistic referential informativeness on cues to common ground (CG) and proposes a process model. Cues to CG provide evidence for CG, that is, for the shared knowledge, beliefs, and attitudes of interlocutors. The presence of cues to CG (e.g., unimpeded listener line of regard or prior mention) is shown to be associated with less informative reference (e.g., pronouns). In contrast, the absence of cues to CG (e.g., impeded listener line of regard or new mention) is shown to be associated with more informative reference (e.g., nouns). Interestingly, informativeness is sensitive to linguistic before nonlinguistic cues to CG (i.e., 2.0 vs. 2.5 years of age, respectively). Reference is cast as a process of active inference, a formulation of Bayesian belief-guided control in biological systems. Child speakers are hierarchical generative models that, characteristically, expect sensory evidence for the evolved, prior Bayesian belief that interlocutor mental states are aligned (i.e., that CG exists). Referential control emerges as an embodied tool to gather evidence for this prior belief. Bottom-up cues to CG elicited by action drive updates to beliefs about CG. In turn, beliefs about CG guide efficient referential control.
... It is posited as a comprehensive theory of non-equilibria in stochastic dynamics [Fri19,PDCF20]. Good overviews of the principle are found in [Fri12] and [RBF18], laying out its mathematical, as well as conceptual, claims; this mathematical content is later examined in detail in [Fri19], especially its third section, and exemplified in a mechanical construction found in [DCFHP21]. Additional worked examples, with commentary on the implied physical dynamics, exist in [AMTB22] and [FHU + 21]. ...
... In this case, we condition the density p(η, µ, s) on the existence of a random dynamical system m, entailing a model of how blanket data is related to external states. This could be thought of as a representation of the blanket, or the model induced by the blanket [Fri12]. Note that the expectation is with respect to the density q(η | µ). ...
... Previous results here established that the recognition density q(η | µ) is a maximum entropy probability density, and that all of the related FEP-theoretic statements about q(η | µ) are both sensible and have statements in the language of maximum entropy. It is initially clear that we can derive the free energy principle from maximum entropy at a conceptual level, given the fact that the free energy principle aims to produce an attractor defining a system, and claims that the system self-evidences to inhabit that region of state space [Fri12,RBF18]. Indeed, the duality between free energy over beliefs and entropy over beliefs is a bit trivial, consisting of (in Proposition 4.1) a negation and a compensatory maximisation. ...
Preprint
Full-text available
In this paper, a simple case of Bayesian mechanics under the free energy principle is formulated in axiomatic terms. We argue that any dynamical system with constraints on its dynamics necessarily looks as though it is performing inference against these constraints, and that in a non-isolated system, such constraints imply external environmental variables embedding the system. Using aspects of classical dynamical systems theory in statistical mechanics, we show that this inference is equivalent to a gradient ascent on the Shannon entropy functional, recovering an approximate Bayesian inference under a locally ergodic probability measure on the state space. We also use some geometric notions from dynamical systems theory$\unicode{x2014}$namely, that the constraints constitute a gauge degree of freedom$\unicode{x2014}$to elaborate on how the desire to stay self-organised can be read as a gauge force acting on the system. In doing so, a number of results of independent interest are given. Overall, we provide a related, but alternative, formalism to those driven purely by descriptions of random dynamical systems, and take a further step towards a comprehensive statement of the physics of self-organisation in formal mathematical language.
... It is through the acquisition of resources (e.g., nutrients) that living systems, via the combination of these reactions, are able to maintain the physiological processes which underwrite their ability to remain far from thermodynamic equilibrium with their environments, temporarily flouting the second law of thermodynamics. For biotic self-organising systems, remaining far from thermodynamic equilibrium means remaining alive (Nicolis and Prigogine, 1977;Friston, 2012Friston, , 2019Demirel, 2014). A system, by harvesting resources from its local environment, can fuel the metabolic processes that allow it to (a) generate itself materially (e.g., protein synthesis) and (b) maintain its organisation in the face of environmental perturbation and despite continuous material turnover; these are the respective processes of "self-production" and "self-maintenance, " which form the basis of Maturana and Varela's (1987) notion of "autopoiesis" (see also Gánti, 2003, Chemoton model). ...
... Living systems are self-organising complex adaptive systems. This means that they can maintain themselves in states that are far from thermodynamic equilibrium, temporarily avoiding the dissipation that inevitably results from the tendency for entropy to increase (Nicolis and Prigogine, 1977;Friston, 2012Friston, , 2019Demirel, 2014). One central property of complex systems is that they are hierarchically organised (Simon, 1962). ...
... Such behaviour is orchestrated in a manner that is dependent on internal states that have a certain degree of independence or detachment from current streams of sensory information (Pezzulo, 2008). Internal states may be generally construed as constituting a system's "internal model" that captures environmental dynamics and the effects of its actions on its environment (Neisser, 1976;Rosen, 1985Rosen, /2012Pezzulo, 2008;Friston, 2012;Pezzulo and Levin, 2016;Schulkin and Sterling, 2019). ...
Article
Full-text available
Intelligence in current AI research is measured according to designer-assigned tasks that lack any relevance for an agent itself. As such, tasks and their evaluation reveal a lot more about our intelligence than the possible intelligence of agents that we design and evaluate. As a possible first step in remedying this, this article introduces the notion of “self-concern,” a property of a complex system that describes its tendency to bring about states that are compatible with its continued self-maintenance. Self-concern, as argued, is the foundation of the kind of basic intelligence found across all biological systems, because it reflects any such system's existential task of continued viability. This article aims to cautiously progress a few steps closer to a better understanding of some necessary organisational conditions that are central to self-concern in biological systems. By emulating these conditions in embodied AI, perhaps something like genuine self-concern can be implemented in machines, bringing AI one step closer to its original goal of emulating human-like intelligence.
... The FEP, as we will understand it, employs the mathematical formalism of non-equilibrium steady-state (NESS) systems to model the properties a complex adaptive system must instantiate if it is to preserve its organization over time (Friston, 2012(Friston, , 2013(Friston, , 2019). Any biological system will be able to maintain order within a boundary (modeled as a "Markov blanket, " more on which below), separating the internal states of this system from the external states of its environment. ...
Article
Full-text available
Biological agents can act in ways that express a sensitivity to context-dependent relevance. So far it has proven difficult to engineer this capacity for context-dependent sensitivity to relevance in artificial agents. We give this problem the label the “problem of meaning”. The problem of meaning could be circumvented if artificial intelligence researchers were to design agents based on the assumption of the continuity of life and mind. In this paper, we focus on the proposal made by enactive cognitive scientists to design artificial agents that possess sensorimotor autonomy—stable, self-sustaining patterns of sensorimotor interaction that can ground values, norms and goals necessary for encountering a meaningful environment. More specifically, we consider whether the Free Energy Principle (FEP) can provide formal tools for modeling sensorimotor autonomy. There is currently no consensus on how to understand the relationship between enactive cognitive science and the FEP. However, a number of recent papers have argued that the two frameworks are fundamentally incompatible. Some argue that biological systems exhibit historical path-dependent learning that is absent from systems that minimize free energy. Others have argued that a free energy minimizing system would fail to satisfy a key condition for sensorimotor agency referred to as “interactional asymmetry”. These critics question the claim we defend in this paper that the FEP can be used to formally model autonomy and adaptivity. We will argue it is too soon to conclude that the two frameworks are incompatible. There are undeniable conceptual differences between the two frameworks but in our view each has something important and necessary to offer. The FEP needs enactive cognitive science for the solution it provides to the problem of meaning. Enactive cognitive science needs the FEP to formally model the properties it argues to be constitutive of agency. Our conclusion will be that active inference models based on the FEP provides a way by which scientists can think about how to address the problems of engineering autonomy and adaptivity in artificial agents in formal terms. In the end engaging more closely with this formalism and its further developments will benefit those working within the enactive framework.
... The simplest, most general, and in many ways most natural expression of the FEP, is formulated for the paths of evolution of a particular system. Note that the FEP was originally formulated in terms of probability densities over paths, as opposed to states [61,62]; it is, after all, a way of expressing the principle of stationary action, which tells us about the most likely path that a particle will take under some potential. 12 When we operate in paths-based formulation, we operate in what are known as "generalised coordinates," where the temporal derivatives of a system's flow are considered separately as components of the "generalised states" of the system [63,64]. ...
Preprint
Full-text available
The aim of this paper is to introduce a field of study that has emerged over the last decade, called Bayesian mechanics. Bayesian mechanics is a probabilistic mechanics, comprising tools that enable us to model systems endowed with a particular partition (i.e., into particles), where the internal states (or the trajectories of internal states) of a particular system encode the parameters of beliefs about quantities that characterise the system. These tools allow us to write down mechanical theories for systems that look as if they are estimating posterior probability distributions over the causes of their sensory states, providing a formal language to model the constraints, forces, fields, and potentials that determine how the internal states of such systems move in a space of beliefs (i.e., on a statistical manifold). Here we will review the state of the art in the literature on the free energy principle, distinguishing between three ways in which Bayesian mechanics has been applied to particular systems (i.e., path-tracking, mode-tracking, and mode-matching). We will go on to examine the duality of the free energy principle and the constrained maximum entropy principle, both of which lie at the heart of Bayesian mechanics. We also discuss the implications of this duality for Bayesian mechanics and limitations of current treatments.
... Indeed, one could argue without boundaries there would be no-thing because everything would be the same thing. The "boundary" itself has probabilistic, stochastic properties that change with the progress of time (Friston, 2012). This boundary can be described technically as a Markov blanket. ...
Article
Full-text available
This paper offers theoretical explanations for why "guided touch" or manual touch with verbal communication can be an effective way of treating the body (e.g., chronic pain) and the mind (e.g., emotional disorders). The active inference theory suggests that chronic pain and emotional disorders can be attributed to distorted and exaggerated patterns of interoceptive and proprioceptive inference. We propose that the nature of active inference is abductive. As such, to rectify aberrant active inference processes, we should change the "Rule" of abduction, or the "prior beliefs" entailed by a patient's generative model. This means pre-existing generative models should be replaced with new models. To facilitate such replacement-or updating-the present treatment proposes that we should weaken prior beliefs, especially the one at the top level of hierarchical generative models, thereby altering the sense of agency, and redeploying attention. Then, a new prior belief can be installed through inner communication along with manual touch. The present paper proposes several hypotheses for possible experimental studies. If touch with verbal guidance is proven to be effective, this would demonstrate the relevance of active inference and the implicit prediction model at a behavioral level. Furthermore, it would open new possibilities of employing inner communication interventions, including self-talk training, for a wide range of psychological and physical therapies.
... The Bayesian inference framework enables quantitative models linking sensory mechanisms (i.e., inputs) with functional behaviors (i.e., outputs) in arbitrary classical [32] and quantum [33] systems. This framework has been applied extensively to biological systems [28][29][30][31][74][75][76][77][78]; here, we consider it more generally. The variational free energy (VFE) that is being minimized in Bayesian inference follows out of classical analytical and statistical physics considerations as a unique form of a least action principle. ...
Article
Full-text available
Evolution is full of coevolving systems characterized by complex spatio-temporal interactions that lead to intertwined processes of adaptation. Yet, how adaptation across multiple levels of temporal scales and biological complexity is achieved remains unclear. Here, we formalize how evolutionary multi-scale processing underlying adaptation constitutes a form of metacognition flowing from definitions of metaprocessing in machine learning. We show (1) how the evolution of metacognitive systems can be expected when fitness landscapes vary on multiple time scales, and (2) how multiple time scales emerge during coevolutionary processes of sufficiently complex interactions. After defining a metaprocessor as a regulator with local memory, we prove that metacognition is more energetically efficient than purely object-level cognition when selection operates at multiple timescales in evolution. Furthermore, we show that existing modeling approaches to coadaptation and coevolution—here active inference networks, predator–prey interactions, coupled genetic algorithms, and generative adversarial networks—lead to multiple emergent timescales underlying forms of metacognition. Lastly, we show how coarse-grained structures emerge naturally in any resource-limited system, providing sufficient evidence for metacognitive systems to be a prevalent and vital component of (co-)evolution. Therefore, multi-scale processing is a necessary requirement for many evolutionary scenarios, leading to de facto metacognitive evolutionary outcomes.
Preprint
Full-text available
Bayesian mechanics is a new approach to studying the mathematics and physics of interacting stochastic processes. In this note, we provide a worked example of a physical mechanics for classical objects, which derives from a simple application thereof. We summarise the current state of the art of Bayesian mechanics in doing so. We also give a sketch of its connections to classical chaos, owing to a particular $\mathcal{N}=2$ supersymmetry.
Article
Full-text available
A computational view of how perception and cognition can be modeled as dynamic patterns of transient activity within neural networks.