World Model – “Free Energy” Selections of Perception & Policy

During their lives humans constantly interact with the physical environment, as well as with themselves and others.
World model learning and inference are crucial concepts in brain and cognitive science, as well as in AI and robotics. The outstanding challenges of building a generalpurpose AI needs world modelling and probabilistic inference, needed to realise a brain-like artificial intelligence that can interact naturally with the real world and our society.

---

For the brain, the basic principles explained below are “Homeostasis”, “Consiousness” and “Free Energy” to reduce entropy.

Homeostasis requires nothing more than
– ongoing adjustment of the system’s active states (M) and/or
– inferences about its sensory states (ϕ),
– in accordance with its predictive model (ψ)
– of the external world (Q)
– or vegetative body (Qη), which can be adjusted automatically on the basis of
– ongoing registrations of prediction error (e),
– quantified as free energy (F)
the contextual considerations just reviewed require an
– additional capacity to adjust the precision weighting (ω) of all relevant quantities.

This capacity provides a formal (mechanistic) account of voluntary behavior—of choice. With the above quantities in place, one can describe any self-organizing (i.e., self-evidencing) system with the following dynamics:

The basic scheme: the organism infers hidden external states in terms of expectations thereof (Q) by minimising variational free energy, based on sensory states (φ).
Crucially, the states that are external to the Markov blanket – when it is embodied – include the viscera. In other words, the external (to the nervous system) states of both the lived world and our own bodies have to be inferred on the basis of (exteroceptive and interoceptive) sensory evidence.
The resulting predictions (ψ) recruit active states (M) – i.e., fire proprioceptive and autonomic reflexes – to realise predicted and preferred external states. This process is enabled by selecting precise sensory evidence through optimising precision (ω).

From equation 1, it is evident that there are three ways to reduce free energy or prediction error.
– First, one can act to change sensations, so they match predictions (i.e., action).
– Second, one can change internal representations to produce a better prediction (i.e., perception).
– Finally, one can adjust the precision to optimally match the amplitude of prediction errors.
It is this final optimisation process – mandated by free energy minimisation – that we associate with consciousness per se and the evaluation of free energy that underpins experience.

Consciousness (as opposed to mere homeostasis) is constituted by inferring changes in expected free energy or, more simply, uncertainty about the experienced world and body. Inferred precision is felt uncertainty. Thus, precision increases when things promise to turn out as expected and it decreases when uncertainty prevails.

In conclusion (from Solms, M., and Friston, K. (2018). How and why consciousness arises: some considerations from physics and physiology. J. Conscious. Stud. 25, 202–238.)

Being (and therefore feeling) is ultimately further reducible to resisting entropy – a process that arises naturally from the fact that any ergodic random dynamical system must differentiate itself from its environment (literally come into being) through the formation of a Markov blanket, whereafter it can respond only to its own states, which (through precision optimisation) are felt.

---

From the perspective of neuroscience and human intelligence, the outstanding challenges for artificial intelligence are clearly many.
The particular focus of things like predictive processing and radical
constructivism highlight three key areas.
– The first is confronting the problem of epistemics in world modelling; namely, building objective functions for inference, learning and action that properly (optimally) balance the imperatives to reach goals, while—at the same time—resolving uncertainty about the context in which those goals are attained. From a normative (optimality) perspective, this rests on combining the principles of optimal Bayesian design and decision theory. In other words, scoring the plausibility of policies in terms of their ability to generate the right kind of outcomes that afford the greatest information gain, under constraints afforded by various loss or reward functions. In the (active inference) neurosciences, this is achieved by absorbing goals into prior preferences and to create objective functions that have both pragmatic and epistemic affordance . In machine learning, a similar direction of travel is emerging in the form of generalised Kullback–Leibler divergence and free energy functionals.
– A second key outstanding challenge is the structure learning problem: namely, optimising the structure and form of world models through active engagement with the sensorium. The brain does not start from scratch; rather, the seeds of intelligence and the architecture of cognition are inherently embedded in an infant’s brain network. In order to overcome the limitations of current models,
we need to further elucidate the learning algorithms, models, and functions that humans have acquired during the evolutionary process and that even infants are born with. Specifically, in order to precisely describe variations in human behaviour and to examine the fitness of the underlying world models, it is necessary to examine their scalability and versatility by integrating hierarchical structures in predictive processing, intrinsic motivations driving continuous development, and factors and parameters altering prior distributions in the brain, etc.
– Third challenge is nicely illustrated by the treatment of self modelling and autism on the one hand, and the special role of language on the other. This challenge speaks to artificial intelligence with minimal selfhood that may be necessary
for linguistic (and non-linguistic) communication. From the perspective of world or generative models, this suggests that the generative model should include a hypothesis that ‘‘I am an agent’’. Various classes of human actions and cognitive development have been interpreted within the theoretical foundations of world model learning based on generative models, called FEP. The probabilistic generative model is also an essential factor of world-model in cognitive robotics. Importantly, human agents or advanced robot agents—that evince rich interactions with environments—should be included in any framework, because the manner of interaction may define the limits and nature of intelligence.
So, what licences the complexity of world models that include a model of selfhood. One obvious answer here is to disambiguate between self and other—to provide the necessary context for communication. In turn, this speaks to the potential importance of encultured artificial intelligence, with a special emphasis on the dyadic interactions and the learning of requisite world models.

World model learning is a fundamental mechanism of human and artificial cognitive systems and contributes to a wide range of cognitive capabilities, e.g., pattern recognition, action selection, social cognition, language learning, and reasoning. In contrast to the distinct development of functional intelligent modules—e.g., visual and speech recognition, machine translation systems, which are trained independently for each functional module—a human brain develops as a whole, while interacting with the body and surrounding environments, i.e., the world.
This suggests that we need to explore computational models for world model learning and inference to build both a human-like intelligence and to understand the human brain. By developing models and algorithms and by testing through biological, computational, and robotic experiments, we aspire to a better understanding of the two sides of the same coin; namely, intelligence.