Tag: Science
-
Power of the Invisible
“The Power of the Invisible: The Quantessence of Reality” from Sander Bais There are two possible outcomes: if the result confirms the hypothesis, then you’ve made a measurement. If the result is contrary to the hypothesis, then you’ve made a discovery. Enrico Fermi Quantum Physics is the solid basis of most of our understanding of…
-
Greatness Cannot Be Planned
Why does modern life revolve around objectives? From how science is funded, to improving how children are educated — and nearly everything in-between — our society has become obsessed with a seductive illusion: that greatness results from doggedly measuring improvement in the relentless pursuit of an ambitious goal. In Why Greatness Cannot Be Planned—The Myth of…
-
Is Ockham’s razor losing its edge?
Is Ockham’s razor losing its edge? New perspectives on the principle of model parsimony The preference for simple explanations, known as the parsimony principle, has long guided the development of scientific theories, hypotheses, and models. Yet recent years have seen a number of successes in employing highly complex models for scientific inquiry (e.g., for 3D…
-
Why probability probably doesn’t exist
(but it is useful to act like it does) David Spiegelhalter All of statistics and much of science depends on probability — an astonishing achievement, considering no one’s really sure what it is. In our everyday world, probability probably does not exist — but it is often useful to act as if it does.— David…
-
The unbearable slowness of being …
The unbearable slowness of being: Why do we live at 10 bits/s? This article is about the neural conundrum behind the slowness of human behavior. The information throughput of a human being is about 10 bits/s. In comparison, our sensory systems gather data at ∼109 bits/s. The stark contrast between these numbers remains unexplained and touches…
-
The power of parsimony
“Walking the talk on multi-level interventions: The power of parsimony“ There is strong consensus regarding the need for multi-level interventions (MLIs) to address today’s complex health problems. The greatest public health burdens globally (e.g., chronic and infectious diseases) derive from a mix of causal and contributing factors and are inherently complex, operating interdependently and reflexively…
-
Energy cost of computation: stochastic thermodynamics?
“Is stochastic thermodynamics the key to understanding the energy costs of computation?” The relationship between the thermodynamic and computational properties of physical systems has been a major theoretical interest since at least the 19th century. It has also become of increasing practical importance over the last half-century as the energetic cost of digital devices has…
-
KNOWLEDGE ACQUISITION hindered by KNOWLEDGE ENTROPY DECAY during language model pretraining
This paper describes how a model’s tendency to broadly integrate its parametric knowledge evolves throughout pretraining, and how this behavior affects overall performance, particularly in terms of knowledge acquisition and forgetting. The concept of knowledge entropy is introduced, which quantifies the range of memory sources the model engages with; high knowledge entropy indicates that the…
Surtil : 