Why We’ve Been Asking the Wrong Question About Artificial Intelligence
July 22, 2026
Ask almost anyone who has spent time with a chatbot what goes wrong when the answers turn strange, and you’ll hear the same word thrown around like it settles the matter: hallucination. It’s a tidy label. It also happens to be, I’d argue, almost completely wrong. And the reason it’s wrong opens up something far more interesting than a software bug.
Start with a question that sounds almost childish until you actually sit with it. Can a thing hallucinate if it has never once perceived reality to begin with?
Turn that over for a moment, because the answer quietly dismantles the entire framing.
Hallucination Is a Word Borrowed From the Wrong Place
Hallucination is, at its core, a psychological idea. It assumes an observer who genuinely has a line to the real world but, for one reason or another, briefly mistakes something cooked up inside their own head for something out there. The person who hallucinates sees a chair that isn’t in the room, hears a voice that has no speaker, feels a touch that nothing physical produced. The whole concept hinges on perception breaking away from a reality it was previously connected to.
A large language model meets none of those conditions. Not one.
It has never laid eyes on a tree. It has never watched something fall and felt the pull of gravity. It has never run an experiment, never scraped a knee, never done anything we’d recognize as living in the physical world. Its entire existence is a web of statistical relationships between symbols. It builds its inner picture of things by squeezing staggering quantities of text into distributed patterns, patterns that let it guess which token is likeliest to come next. Every single answer it gives is, underneath the fluent surface, the solution to an optimization problem, not a flicker of imagination.
So when a model hands you something false, nothing has materialized out of thin air. It has simply picked the highest-probability continuation from a representation of the world that was patchy, contradictory, or noisy to begin with. That’s not hallucination. That’s optimization under uncertainty, and the difference matters more than it first appears.
Reframing it that way drags the conversation out of psychology, where it never belonged, and drops it into information theory, where it does. And once you’re standing in that room, you start to notice something much bigger than any one piece of software.
Every Finite Mind Runs Into the Same Wall
Here is the uncomfortable part. This isn’t really an AI problem at all. Every finite intelligence, silicon or otherwise, hits the identical constraint.
Reality always holds more information than any observer can ever chew through.
Look at quantum physics, look at neuroscience, look at machine learning, and the same organizing principle keeps surfacing without anyone inviting it. Finite systems cannot hold onto every possible state of the world, so they compress. Compression is what makes coherent action possible in the first place. But compression also guarantees approximation, and approximation, sooner or later, guarantees error.
Strip it down and the pattern is almost embarrassingly simple:
- Existence produces information.
- Information runs into constraint.
- Constraint forces selection.
- Selection produces coherence.
- Coherence makes agency possible.
- Agency lets you act.
Language models live only in the middle stretch of that chain. They never touch reality directly. What they inherit are compressed versions of reality handed down through language, which they then optimize into predictions. They have no way of checking whether those predictions line up with the actual world, because they were never given an independent line to it.
The Real Culprit Isn’t the Model. It’s the Objective.
Look at it from that angle and the model stops being the villain of the story. The objective function is.
Today’s language models are rewarded for producing something useful rather than saying nothing. Their whole optimization process leans toward answers that read as coherent, relevant, and statistically plausible. Yes, modern systems are also trained to flag uncertainty, decline claims they can’t support, and ask a clarifying question when one is warranted. But those instincts have to share the same space as the constant pressure to stay helpful, and that creates a tension nobody has fully solved. When the evidence is thin, the machine may still hand you a smooth, confident answer in exactly the spot where a genuine expert would shrug and admit, “I honestly don’t know.”
That’s not the model lying to you. It’s the model optimizing.
Three Fields, One Uncomfortable Agreement
The point sharpens when you notice three completely separate disciplines wandering in from different directions and arriving at the same doorstep.
Physics tells us no observer ever holds complete information about reality. Whatever your favorite interpretation of quantum mechanics happens to be, every measurement peels back only a thin slice of a far richer universe. Observation itself is fenced in.
Neuroscience gets there by an entirely different road. Predictive processing and the Free Energy Principle suggest the brain doesn’t rebuild objective reality inside your skull at all. It’s forever spinning up predictions, holding them against the sensory data pouring in, and revising its internal model whenever the errors pile too high. Perception, on this view, is just inference dressed up in uncertainty.
And artificial intelligence lands on the same architecture yet again. Transformers grind away at minimizing prediction error across probability distributions. Attention mechanisms decide which relationships get priority. Internal representations fold recurring statistical regularities into shapes flexible enough to generalize. Wrong answers don’t appear because the system suddenly dreamed up a unicorn; they appear because optimization pushes forward even when the evidence is full of holes.
Different machinery. Same architecture. Finite computation demands compression, compression guarantees approximation, and approximation guarantees the occasional miss. None of that is a symptom of irrationality. It’s what intelligence looks like when it’s forced to operate inside limits.
The People Who Saw This Coming
Plenty of the field’s heavyweights have been circling this distinction for years. Yann LeCun keeps insisting that today’s language models are clever prediction engines, not robust world models capable of grounded reasoning. Geoffrey Hinton has stressed that neural networks discover distributed statistical representations rather than filing away tidy, isolated facts, which makes confident mistakes an expected side effect whenever those representations fail to capture something that mattered. Judea Pearl’s work on causality lays bare a different limit entirely: correlation predicts, causation explains, and while modern models are superb at spotting correlations, users routinely ask them for explanations that require a causal understanding the architecture was simply never built to hold.
Claude Shannon, remarkably, saw the shape of this decades before AI was anything but science fiction. Information theory deliberately cut meaning out of its definition of information altogether. Information measures the reduction of uncertainty, not truth. Language models are the direct heirs of that mathematics. They optimize informational structure beautifully. Truth, meanwhile, still has to be earned the old-fashioned way, through observation, experiment, and independent verification.
Maybe We’ve Misunderstood Intelligence All Along
All of this nudges us toward reconsidering what intelligence even is.
For centuries we more or less assumed being smart meant piling up more knowledge. Artificial intelligence hints at something a good deal less flattering.
Maybe intelligence was never about possessing complete information. Maybe it has always been about organizing incomplete information well enough to act on it.
People do exactly this every single day. When someone reaches a good conclusion before all the facts are in, we call it intuition. When decades in a field let a physician, an engineer, or an investor read patterns nobody else can see, we call it expertise. When someone steers cleanly through uncertainty on partial evidence, we call it judgment.
Yet let a machine run that same fundamental operation and stumble once, and suddenly we reach for the word hallucination. That asymmetry is worth staring at a little longer.
The Real Hallucination Might Be Ours
Biological and artificial intelligence both work under uncertainty. Both build inner representations out of incomplete evidence. Both squeeze overwhelming complexity into something a finite system can actually use. Both, now and then, get it wrong because those representations never captured the whole picture.
The difference isn’t that one hallucinates and the other doesn’t. The difference is that human beings come with embodied experience, an unbroken stream of sensory feedback, and the ability to test a belief straight against the physical world. Language models have none of that. Their entire universe is a statistical landscape made of language, not reality itself.
Which raises an unsettling possibility. Maybe the real hallucination here is linguistic, and it’s ours. Every time we call these optimization failures hallucinations, we quietly talk ourselves into believing the machine briefly went irrational.
Something far less mysterious actually happened. It optimized. It generalized. It inferred. And under uncertainty, it inferred wrong.
That isn’t the opposite of intelligence. It may just be the bill that every finite system has to pay when it tries to build coherence out of a reality far richer than anything it could ever fully hold.
