Information theory: A foundation for complexity science

Amos Golan and John Harte published a perspective paper, consolidating the insights and research on knowledge and models from incomplete information in complex environments, based on MaxEnt

Modeling and inference are central to most areas of science and especially to evolving and complex systems.
Critically, the information we have is often uncertain and insufficient, resulting in an underdetermined inference problem; multiple inferences, models, and theories are consistent with available information.
Information theory (in particular, the maximum information entropy formalism) provides a way to deal with such complexity. It has been applied to numerous problems, within and across many disciplines, over the last few decades.
In this perspective, we review the historical development of this procedure, provide an overview of the many applications of maximum entropy and its extensions to complex systems, and discuss in more detail some recent advances in constructing comprehensive theory based on this inference procedure.

The mathematical architecture of the METE. Empirically testable metrics, such as the distributions of abundances over species and metabolic rates over individuals, the species–area and endemics–area relationships, and an energy-equivalence principle derive from specified mathematical operations on the two fundamental distributions in the theory: an ecological structure function, R, and a spatial distribution, Π, which in turn, are derived using MaxEnt.

We also discuss efforts at the frontier of information-theoretic inference: application to complex dynamic systems with time-varying constraints, such as highly disturbed ecosystems or rapidly changing economies.

Overview of representative applications of information-theoretic MaxEnt inference

With all problems in science, the more information we have, the better will be the models and theories we can construct. However, for hugely complex systems, like the ones we confront in economics and ecology, we never will have enough information to unambiguously predict outcomes.

For such systems, theory and model construction are an underdetermined inference problem. If, however, we specify available information as constraints and build directly on the MaxEnt principle, we ensure that out of all possible models that are consistent with the information we have, the chosen one is least biased.

Both within each of us and surrounding all of us are systems of immense complexity. Critical to our survival and well-being is understanding those systems sufficiently well so that we can make reliable predictions and design effective interventions. MaxEnt, an information-theoretic method of inference, provides a powerful foundation for the study of complex and continuously evolving systems. It is a powerful method for extracting insight from sparse, uncertain, and heterogeneous information and is also a foundation for complex systems theory construction. We expect applications of MaxEnt inference to continue expanding the frontiers of complexity science within and across disciplines.

One response to “Information theory: A foundation for complexity science”

  1. […] ecosystemen leren ons de risico’s van faling door onbeslistheid of rigiditeit, ten gevolge van de verkeerde evenwichten tussen veerkracht en efficiëntie. De informele communicatie (koffie..) is vaak de overleving om het te veel aan efficiëntie kort te […]


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: