He was drafted against his will. Ordered to hunt down the same guerrilla fighters he made part of just months before.

Now, he found himself deep in the wilderness doing practice drills with his new squadron, when he suddenly spotted rustling in the bushes. His heart pounding in his throat, he saw a formation of camouflaged guerrillas looming with AK-47s ready to burst. Instinctively, he raised his rifle, flipped the safety and aimed.

“Don’t shoot.” A hand on his shoulder. “It’s just a boy.” He lowered the rifle, looked again, and was dumbfounded by what he saw: a ten-year old boy herding cows. Not with an AK-47, but with a stick.

The soldier shared his story with neuroscientist Lisa Feldman-Barrett, who recounts it in her book. He was desperate to understand how he could have mis-seen what was right in front of him and nearly killed a child. “What is wrong with my brain?”

There was nothing wrong with his brain. It had worked exactly as it should have.

The red pill

How would you react if I told you your day-to-day experience is, in fact, a carefully controlled hallucination?

This would sound familiar if you've seen The Matrix. But what if this was science’s version of your reality?

One of the great emerging theories in neuroscience is predictive processing. It asserts that the brain predicts large swaths of experience from memory before it happens, rather than construct it from what you see, smell, hear and feel as it happens. This saves energy that can be focused as attention on new, uncertain elements in the environment.

The soldier’s brain predicted guerrilla fighters from matching earlier experiences so he could react quicker. If it really had been a guerrilla, the seconds he won predicting the scene rather than reacting after fully perceiving could have saved his life. The prediction was wrong but killing the child as a result — tragic as it would have been — wouldn’t hurt him biologically. The predictive brain wins at natural selection with metabolic efficiency.

The Predictive Brain

From the day you’re born to the day you draw your last breath, your brain is trapped in a dark, silent box — your skull. Blind and deaf, it depends on data collected by sensors to know what's going on inside its body and the world outside:

• Extero-ceptive sensors — Eyes, ears, nose, skin and tongue bring in sense data from the world around you.
• Intero-ceptive sensors — Neural pathways bring in sense data from within the body.

Mathematical soup / Entropy

This sense data isn't actionable. It doesn’t come in as meaningful sights, smells, sounds and touches, but as random scraps light waves, chemicals, vibrations and air pressure changes. It's reality rendered as numbers without intrinsic meaning: quantitative.

The brain’s job is to interpret sense data in context of the environment so it can keep you alive and well, i.e. avoid you falling down the stairs, getting dehydrated or becoming lion lunch. In context, data gains meaning and becomes information. As per information theory pioneer Claude Shannon, information “resolves uncertainty”. In doing so, it informs agency: your brain knows what to do next.

• Data — My body records a drop in temperature and increasing friction from wind. Information — My brain interprets the change as a sign of bad weather to come. Action — I put on a rain jacket.
• Data — Eyes record image of a lion near the river. Information — I’ll be in danger if I continue my path. Action — I head to another water source.

Memory

The brain faces an inverse-inference problem: how does it figure out what's going on (causes) from random data (effects)? It solves this by drawing on its lifetime database of past experiences: your memory.

Inside the skull, the brain asks itself what happened last time it was in a similar situation. "What are these wavelengths of light most like?" The answer need not be perfect, just close enough to inform effective action.

The Beholder's Share

Chaos becomes structured and navigable through mental maps modelled from past experiences. We make sense of new things through what we already know. Inversely, we can't make sense of data we don't have mental maps for.

Take a look at these three visuals. You probably see fairly meaningless black lines. Now read what they are meant to represent below, then look at the visuals again. They'll mean something now. Not because they changed, but because you have.

The conceptual artist Marcel Duchamp famously said an artist does only 50% of the work in creating art. The other 50% is done by the viewer's brain.

“The creative act is not performed by the artist alone; the spectator brings the work in contact with the external world by deciphering and interpreting its inner qualifications and thus adds his contribution to the creative act.” — Marcel Duchamp

Mental LEGOs

Such pattern-matching follows from the brain’s capacity for abstraction. Experiences are not so much stored in full, but rather compressed into concepts: LEGO-like mental building blocks that can be mixed and matched with other concepts across contexts.

This how you intuit flowers, watches and wine as “gifts” even if the objects look nothing alike. Different realities with the same meaning in a particular context. Inversely, the same reality gains different meaning in different contexts: a glass of wine can mean blood of Christ, a toast and a cosy evening with friends.

Functional maps

Abstraction summarises physical realities to their functional features. How something looks, sounds and smells is abstracted away; stripped down to what it can do in a given context. These become models of meaning for the brain to project onto other and even entirely new realities so it can pilot them quicker. Past experience is focused for function to resolve present ambiguity for action.

• Flowers, wine, watches can all be "gifts" in festive contexts.
• Umbrella, coat, plastic bag, car, apartment and a mindset to not care can all "protect from rain."
• Salt, barley, little shells, copper coins, paper notes, mortgages, Bitcoin have all been used as "currency" in context of transacting goods and services.

The brain doesn't project past experiences in full, but rather recombines conceptual pieces (LEGOs) from across its memory database in novel ways. This is how you can instantly navigate situations you've never encountered before, without having to think about it.

Imagination

It's abstraction that unlocks imagination: mental modelling that builds on top of memory. The brain playfully remixes mental LEGOs to create new, imagined realities. Imagination orients creativity and pulls it in a particular direction.

• Memory is accumulated mental modelling built on feedback from the outer environment. It's past experience stored as what the brain thinks is true and guides present experience accordingly.
• Imagination playfully builds on memory to create scenarios for the future. It drives creativity and, more than memory, pulls attention in the direction of the vision that is imagined.

The Predictive Brain

Scientists used to believe sense-making happened reactively bottom-up. Data moves in through senses and travels up the nervous system into the brain, where it gets interpreted so we can act. Senses record, brain processes, body re-acts — in that order.

Such an input-reaction sequence seemingly matches how we experience the world. For example, the brain's visual system feels like it works like a camera: eyes record visual information and the brain processes it into an image. Hence, the soldier believed something was wrong with his brain. How else do you explain mis-seeing a boy herding cows for guerrillas with machine guns?

The picture changes in the face of survival. Remember, the brain's job is to keep you alive and well. The constant influx of ambiguous data is cognitively overwhelming. If the brain were to reflexively interpret all of it from the ground up, it'd drown in uncertainty and be (too) slow to act. The soldier would get shot before he'd understand he's in danger.

To avoid analysis paralysis, the brain takes a shortcut: rather than process all the sense data, it pattern-matches with memory to predict what is most likely to happen next. "Last time I was in a situation like this, what did I see, hear and feel next?" That prediction is constructed as experience.

The soldier mistook the child for a guerrilla because that's what happened next last time the bushes rustled when he was:

• in the woods
• with his comrades
• holding a rifle
• his heart pounding

From memory, the brain (1) defines random data and (2) fills in blind spots to proactively create a meaningful experience that guides action. How you see the world is not a photograph, but a construction of your brain so fluid and convincing that it feels like it is. Lightwaves, air pressure changes and chemicals are predictively constructed as objects, sounds, tastes and smells based on what the brain believes most probable from past experience.

In case it hasn't fully sunk in yet: prediction is to be taken literally. Yes, you sense changes before they actually happen. Most of the time predictions are accurate and match reality. You look at cows and see cows. But have you ever seen a friend's face in a crowd when they weren't actually there? Ever sworn your phone vibrated when it didn't? You really did see and feel those things. They were sensory anticipations then invalidated by reality. Prediction errors.

Same is true for experience of your body. Changes are sensed before the relevant data from organs and hormones arrives.

• When you're thirsty and drink water, that thirst seems to subside at once. In reality, water takes about 20 minutes to hit your bloodstream.
• Research verified the expectation of having consumed caffeine alone improved attention, cognitive performance and mood.
• Turns out Pavlov's dogs didn't salivate as a reaction to the bell, but rather (their brains) anticipated the experience of food from memory of the bell and prepared the body for eating. Similarly, if you'd imagine your favourite right now, scans would detect increased activity in brain regions associated with taste and smell — which trigger salivation.

At any given time, the brain experientially represents what it believes is going on in body and world, and, from there, computes what is most likely to happen next. It does this according to Bayes' theorem, which describes the mathematical probability of an event from prior knowledge of variables related to that events.

Prediction errors

Each time a prediction proves true in a context, the brain attributes a higher probability to it and is more likely to re-project it in the future. Every once in a while however, incoming sense data unexpectedly contradicts the prediction. Guerrillas turn out to be cows and the friend isn't really there. The brain corrects the experience, our attention suddenly shifts and we feel surprised. We learn: reality forces the mental map to update. The correction will inform more accurate predictions going forward.

Prediction beats reaction

The brain predicts and then corrects rather than detects and reacts because it reduces uncertainty and costs less energy. A reactive brain would have to compute everything on the spot from scratch every single moment. It'd be too ineffective and inefficient to be naturally selected.

The predictive brain follows Karl Friston's free energy principle, which says that organisms aim to minimise energy use so to maintain homeostasis: a state of internal biological balance.

We can move faster through territories using maps we already have rather than constantly drawing new ones on the go. By assuming known parts of the territory, the brain saves energy it can focus as attention on new, uncertain elements — those we are not sure to be safe yet.

The map predicts the territory

Of course, as the saying goes, the map is not the territory. Turns out it doesn’t have to be: it just need to approximate the territory well enough to be practical. When the map errors and the territory hits us in the face, the map updates so the brain can predict better next time.

Crucially, the map is bound to the territory. What is true in some situations might not be in others, which is what we mean when we say "it's relative." Different contexts have different truths and require different mental models to navigate effectively. When we predict from an old map in a new territory, we trip and fall: prediction errors. The brain learns and fills in the blind spots.

As trial-and-error adapts mental maps to mirror differences across territories, they become dynamic and context-independent. Predictions from these models are true in a greater variety of contexts, making them more probabilistically sound. When truths achieve seemingly universal predictive reach, we like to call them wisdoms.

Novelty is costly

The free energy principle implies the brain innately avoids novelty for as long as it can afford to. Because correcting internal models is metabolically costly, uncertain or contradicting data is purposefully left out in constructed experience. The brain doesn't predict what it doesn't know and it doesn't map what it doesn't have to, to save energy. You never see the full picture of reality, only the sliver the brain has had to learn about in the past.

The brain only corrects its models if the future risk of not doing so warrants the cost. The metabolic investment needs to pay off.

• If your model predicts that lions are friendly and one almost bites off your arm because you tried petting it at the zoo, your brain will update the model so you are more careful next time.
• People who believe they don't need sun-screen only do so until they get severely sun-burnt.
• frat guys

We predict what the world will be like based on what we already know and the world adjusts those predictions by feeding back where they're wrong. However, updates cost energy and so the brain by default avoids being proven wrong, sticking to what it knows. To learn beyond what is biologically necessary therefore takes intention, the willingness to expose our brains to information that is new or conflicting. You need to actively seek to be proven wrong by confronting your assumptions across contexts.

Experiential blindness

If your brain doesn't have concepts to anticipate and interpret sense data, you are experientially blind to it.

• Unless you already had concepts, the visuals shown earlier looked like meaningless black lines at first. Once you were handed concepts, you saw something in them. The visual stays the same, it's your brain that changed and now adds its prediction.
• A spoken language you don't know sounds like gibberish. Once you learn the language, the same sounds you didn't know what to make of earlier all of a sudden gain meaning.
• Some people that are born blind can gain eyesight through operation. Still, for days, weeks or even months they remain blind to parts of reality because the brain doesn't know how to experientially map them.

We all see the world through our unique prisms, modelled after the unique mosaic of (1) our own lived experiences and (2) those of others we are subjected to through relations and media (movies, books, timelines).

And so two people looking at the same reality can come to wildly different conclusions about it. The more different the information ecologies they've lived in, the more different their conceptual maps and the more they'll struggle to find common ground. They're experientially blind to each other.

It explains why conversations about politics and religion are mostly dead on arrival. These are games of language with no cost to being wrong. Each participant can simply keep arguing from the different concepts modelled from their different pasts since there are no negative consequences for well-being and no prediction errors. Not changing your mind is the path of least metabolic resistance.

Echo chambers

Each time reality doesn't correct predictions, the underlying models are judged effective and assigned a higher probability for future prediction. Predicting present from past, thinking patterns self-reinforce and become habitual over time. We naturally see what we want to see, hear what we want to hear, and believe what we want to believe.

Misguided predictions in nature cost you so you must change your mind to survive. In modern environments however, most risks have been designed away so prediction errors don't occur. You can be very wrong for a very long time and be perfectly fine throughout. Naturally selected for metabolic efficiency, brains are wired to get stuck in echo chambers.

Echo chambers

Baby brain development demonstrates the mechanics of predictive processing. It makes for a good summary before moving on.

A baby's experience is pure sensation. They're overwhelmed by sense data they don't know how to deal with because they have no memory to predict from. Their world is uncertain and chaotic.

As the baby crawls, grabs and plays, its brain maps sensations to the environment and, through trial and error, learns what it should and shouldn't do. Thanks to sensory feedback, it learns that the couch is soft, the floor hard and the dog kind. When experience doesn't resolve uncertainty, the baby turns to mom and dad. When they are calm, the baby calms too. Parents regulate their baby's emotional state with soothing touches, looks and sounds.

This is why caregivers are crucial to baby brain development. By contextualising sense data, they help the baby form its first mental maps to predict from. These maps make the uncertain navigable so the baby brain gains control and agency. As the baby crawls further, its models refine through learning from prediction errors.

Meanwhile, its library of mental LEGOs compound

Not having memory to predict from is one reason babies cry and sleep so much. The constant stream of ambiguous sense data and prediction errors exhausts the brain. In contrast, adult brains smooth-sail in familiar environments because every detail can be predicted without a cognitive sweat. But, like babies, they too tire quickly in unmapped exotic environments, as happens with culture shock.