The Free Energy Principle Made Clear

Curt explains Karl Friston's weltanschauung in less than 1,000 words.

The Free Energy Principle: a “theory of everything” that includes not only brains, societies, but perhaps even the universe itself…

It’s known for being considerably convoluted but the principles underlying it are actually straightforward.

If the FEP is right, then is your entire reality a ‘controlled hallucination’? What does that even mean? And how does this relate to entropy?

Below is the core idea, stripped of the (intimidating) math.

Imagine you’re in a completely dark room. You can’t see anything. But you still have expectations. You predict where the walls are, where the furniture may be. Now you see that your brain doesn’t merely receive but infers.

This is the difference between passivity and activity.

Your brain is continually and unceasingly building a model of the world. Then, after that model has been created, it checks this against incoming sensory data. When there’s a mismatch, the brain adjusts -- either its model, or its actions, or its own inner states.

You feel this mismatch as the feeling of surprise. This minimization extends even to non-conscious systems (mentioned in later paragraphs).

Bridging Perception and Reality

That “mismatch” is what Friston calls “free energy” -- but it’s not the same as the free energy in physics (though there are connections). Think of it more like “uncertainty” or, as referenced in the prior paragraph, “surprise.”

Mathematically, it’s often represented as the difference between what your brain expects to sense, and what it actually senses. Formally, it looks roughly like this:

\(F = D_{KL}[Q(\text{world}) || P(\text{world}|\text{brain})]\)

where D_KL is the Kullback-Leibler divergence, a measure of how different two probability distributions are. Q(world) is your brain’s internal model, and P(world|brain) is the “true” distribution, given your brain’s state. The smaller the difference, the lower the free energy.

But why does the brain despise surprise? Because survival. A creature that can predict where the predator will be next is a creature that lives another day. A creature that can anticipate its own needs (hunger, thirst) before they become critical is a creature that (relatively) thrives.

You may say “Curt, I love surprise. I dislike humdrum movies with tedious plots. I like rollercoasters. Chains and whips excite me.”

Well, firstly, TMI.

Secondly, the reason you like this is that you actually dislike OVERALL floods of surprise, and you know (deep down… or at least your brain knows) that transient planned and (relatively) diminutive surprise, actually teaches you. You learn from them. Thus you’re less likely to be daunted by an unpredictable paralyzing surprise in the future.

Navigating the Free Energy Landscape

So, the FEP says that everything -- from perception to action to internal homeostasis -- is about minimizing this “free energy,” this “surprise.” It does this through something called a “gradient descent.” The standard example is to think of a ball rolling downhill.

You can endow this ball with intent, if you like, and think of it always “seeking” the lowest point it can get to.

Similarly, your brain is always “rolling” toward the lowest free energy state. Mathematically, this is expressed as a flow on a “free energy landscape” which can be (roughly) written as:

\(\dot{\alpha}(\tau) = (Q_{aa} - \Gamma_{a}) \nabla_{a}F(s, \alpha) + \ldots\)

See that ∇? That’s the “gradient” – the direction of steepest descent. There are AI algorithms that use this exact approach. The Q and Γ terms represent the system’s internal dynamics… basically, the “shape” of the landscape it’s navigating.

Note: You can always equivalently reformulate the least resistance principle by inverting it into a principle of maximal resistance. This latter model is what I employ when my wife makes me spend time with people.

The Brain’s Predictive Dance

Now let’s get to this “controlled hallucination”. It means your experience isn’t a direct reflection of “reality” (whatever that is). It’s a construction, built from your brain’s best guess about what’s out there, constantly updated by sensory data… but always, always a guess. It’s a forecast.

Your brain is a weatherman except it’s more like a “world-man” (or world-woman). It’s predicting what the world is like. And instead of “is there sun tomorrow?” it’s “is this Justin Bieber playing?” or “It is! Which song is it?” (an instant from now. and then another prediction an instant from then. and so on. Relentlessly. This is your moment of the “now”) The next question is naturally “which of Dante’s circles must I be in?”

No need to get solipsistic on me. The FEP isn’t saying reality is fake. It’s saying your experience of reality is an active, predictive process, not a passive reception. You’re not a blank canvas only receiving imprints… Your brain is a scientist of sorts, constantly testing hypotheses.

Scale-Free Principles and Entropy

Friston and others argue that the FEP isn’t just about brains. It applies to any system that persists over time – a cell, an organism, a society, likely even the universe itself. This is what’s meant by “scale-free” – the principle works the same, regardless of size or complexity. There’s even connections to the laws of physics via thermodynamics / statistical mechanics, as the FEP deals intimately with entropy.

Entropy is NOT a measure of disorder. That’s a foolish imprecise manner of thinking of entropy that has somehow permeated its way to the public via the likes of science communicators, but that’s not what entropy is.

However, living creatures, brains included, are (mostly) low entropy systems. To keep that low entropy, they predict their environment and act to maintain their internal states within narrow bounds.

A cell maintains its internal environment that it believes is best for its survival. So, by minimizing “surprise,” a system is, indirectly, minimizing its own entropy.

I want to hear from you in the Substack comment section below. I read each and every response.

- Curt Jaimungal