“The Primacy of Doubt“, By Tim Palmer discusses topics from climate change to quantum physics, how the science of uncertainty can help predict and understand our chaotic world.
Tim Palmer brings us his first foray into popular science writing with a carefully considered and often expert exposition on a vast range of subjects. The credo of Palmer’s book is that the scientific concept of uncertainty deserves higher status in public understanding,
philosophy and culture. He demonstrates this with concrete examples in the fields of fundamental physical theory, weather, economics and religious philosophy.
The key idea discussed is about how the geometry of chaos can explain our uncertain world, from weather and pandemics to quantum physics and free will.
Tim Palmer delivered a talk with key ideas at the Royal Institution on 21 April 2023 (recording available on Youtube). He explores how it provides the means to predict the world around us, and provides new insights into some of the most astonishing aspects of our universe and ourselves.
Chaos theory and geometry: can they predict our world?
The book itself
- explores possible insights provided by the geometry of chaos into consciousness, free will, and moral responsibility
- charts the discovery of the remarkable fractal geometry of chaos and its links to some of the deepest theorems of modern mathematics
- explains how the geometry of chaos allows us to understand why systems can be predictable most of the time, but spectacularly unpredictable on occasion
- describes the development of ensemble prediction systems for predicting weather, climate, economics, pandemics, and human conflict
- illustrates how these ensemble systems allow us to make better decisions
Most of the book is based on the great insights and evolutions made in weather prediction, discussed in the 2011 Dennis Sciama Memorial Lecture (Lorenz Gödel and Penrose: new perspectives on determinism and unpredictability, from fundamental physics to the science of climate change) and published in articles like e.g.
- 2011: Uncertainty in weather and climate prediction
- 2014: Lorenz, Gödel and Penrose: new perspectives on determinism and causality in fundamental physics
- 2017: The primacy of doubt: Evolution of numerical weather prediction from determinism to probability
- 2019: The ECMWF ensemble prediction system: Looking back (more than) 25 years and projecting forward 25 years
(a) High predictability and therefore a high level of confidence in the transition to a different ‘weather’ regime.
(b) A high level of predictability in the near term but then increasing uncertainty later in the forecast with a modest probability of a transition to a different ‘weather’ regime.
(c) A forecast starting near the transition point between regimes is highly uncertain.
In chaotic systems, some states are very predictable, others not.
Ensembles of forecasts can provide estimates of state-dependent predictability.
The book touches the “Butterfly Effect“ and its variations, also explained as
(1) butterfly effects of the first kind represent the sensitive dependence of solutions on initial conditions (SDIC);
(2) butterfly effects of the second kind represent the hypothetical role of initial tiny perturbations in producing an organized large-scale system at large distances (e.g., a tornado);
(3) butterfly effects of the third kind, or the so-called real butterfly effect, represent the role of small scale processes (within a highly turbulent region in contributing to the finite predictability of large scale processes in.
The three kinds of butterfly effects are not exactly the same.
| Kind | Features | Consequences | Acceptance |
| 1st | SDIC | Finite Predictability | Well accepted |
| 2nd | Butterfly’s flap | Formation of a tornado | As a metaphor |
| 3d | (1) Collective contribution of turbulent small-scale processes (2) Ill-conditioning and numerical instability | Finite Predictability | On-going |
The blue lines show the trajectories of the individual forecasts that diverge from each other owing to uncertainties in the initial conditions and in the representation of sub-gridscale processes in the model. The dashed, lighter blue envelope represents the range of possible states that the real atmosphere could encompass and the solid, dark blue envelope represents the range of states sampled by the model predictions.
The phrase ‘‘primacy of doubt’’ is taken from James Gleick’s biography of the great theoretical physicist Richard Feynman:
‘‘He believed in the primacy of doubt:
not as a blemish on our ability to know,
but as the essence of knowing.’’
In the final section of the book, Palmer’s own theory of fractal state-space geometry and determinism, called the Invariant Set Postulate, is brought to the fore. He proposes that there exists a state-space for the entire Universe. This is an abstract object within which each point defines everything about the Universe in all its space, time and in all possible variations. Assuming such a statespace exists, and that the Universe is fundamentally chaotic, he argues that the set of points in this state-space which represent what occurred in the real world are likely to have a fractal geometry, just like the weather.
A great summary of the book can be found in the well known traditional:
“For want of a nail, the shoe was lost.
For want of a shoe, the horse was lost.
For want of a horse, the rider was lost.
For want of a rider, the battle was lost.
For want of a battle, the kingdom was lost.
And all for the want of a horseshoe nail”.
Surtil : 