AI's 'agents' are misnamed: autonomy requires computational closure, not compute stacking
Determining autonomy is not about Turing completeness, but whether a system's internal loops can close on themselves—which is exactly where today's so-called AI agents fall short.
The video won't play here. Listen to the audio instead:
The argument · tap a timestamp to hear it
Not all state changes and information transfer constitute computation
Not every information transfer or state change counts as computation. Nima developed a physical computation theory to provide a falsifiable definition, in contrast to casual neuroscience papers where 'that circuit computes X' often functions as an unjustified assertion.
— Nima DehghaniTuring completeness does not guarantee autonomy
A French team demonstrated this with oil-droplet walkers: liquid droplets bounce on vibrating oil surfaces, with ripples feeding back to create chaotic trajectories with memory, erasure, and write primitives—Turing-complete, yet not autonomous. The closure comes from the experimenter flipping the wave phase externally, not from the system itself. This gap led Nima to develop the concept of computational closure: closure must emerge from within the system.
— Nima DehghaniCalling AI systems 'agents' is a fundamental misnaming
Agency emerges from computational closure—when a system samples the environment, constructs an internal world model, and uses it to regulate its own behavior and state (like biological homeostatic setpoints). Today's AI systems called 'agents' lack this loop entirely. When they fail, it's described as the system 'going out of control,' but that gets the causality wrong—they didn't develop agency and rebel; humans simply didn't set the right boundaries.
— Nima DehghaniBiology cannot afford backpropagation's weight-update machinery
Feedforward networks are straightforward to design and stack, but recurrent networks are another matter—Nima calls them 'neuroscience's holy grail.' While the brain must constantly change its operations, it cannot afford the rapid, large-scale weight updates that backpropagation requires. Nima breaks the problem into two basic building blocks: 'funnels' (systems with final states) and 'gears' (systems where states can reverse), using this framework to analyze what divisive normalization and winner-take-all circuits actually compute.
— Nima DehghaniTwo circuits combined can compute what each cannot alone
Divisive normalization in isolation produces only a limited set of output states, as does winner-take-all. Connect them in different directions—controlling which pushes which, where inputs flow—and the total state space expands dramatically. Even more striking: purely 'funnel'-based systems, when combined this way, suddenly produce 'gear'-like reversible behavior that neither component could achieve alone. Nima sees this as how evolution reuses modules, and suggests a path forward for designing recurrent systems based on mathematical prediction of combination effects, rather than relying on massive datasets and backpropagation.
— Nima DehghaniFeedforward networks face mathematical barriers some problems cannot overcome
Nima is preparing a paper (already at the sixteenth revision) proving that certain physical problems are mathematically unsolvable by feedforward systems—they require recurrent architecture. He connects this to embodied knowledge: large language models can describe rain's temperature, paint it in Monet's style, write haiku about it, yet experience no rain. His dog never studied Galileo's physics but genuinely knows what rain is. This lived experience is what feedforward systems structurally cannot access; recurrence can.
— Nima DehghaniRecurrent systems can learn without changing weights
Nima and collaborator Bahtash Babadi are completing a paper showing how a fully recurrent system can explicitly implement all the mathematics backpropagation performs—but without changing any weights. Learning happens in the system's recursive state rather than through weight updates. Nima is careful to note this is not a claim about how brains actually work, but rather a pointer toward where investigation should focus.
— Nima DehghaniNot all brain activity is computation—some is information relay
When asked what fraction of brain processing is actual computation, Nima says he doesn't know. But he invokes David Deutsch's standard: good explanations must be difficult to arbitrarily patch when new phenomena appear. Using his current work on E. coli chemotaxis as an example, if he discovers that some component is merely information relay, he'll say so directly—rather than forcing the 'computation' label onto every corner of the system.
— Nima DehghaniIn their own words · checked verbatim
If you have a theory, you can change your theory to accommodate everything. That's not a good theory, right?
Nima Dehghani0:00
Not every information transfer is a computation.
Nima Dehghani21:14
Because it's the experimentalist who goes there and switches the action over there that closes the loop, right?
Nima Dehghani37:59
Oh, yeah. That's such a misnomer. It's terrible.
Nima Dehghani44:04
The agency comes from this aspect of having this sort of computational closure built into this system that now you can have an understanding of the environment.
Nima Dehghani46:08
Recurrence is a beast, right? And I think that's the holy grail of neuroscience.
Nima Dehghani1:06:00
it has no experience of what RAIN is. Like, you know, it just absolutely doesn't know what RAIN is. My dog knows what RAIN is.
Nima Dehghani1:20:24
Glossary
- computational closure
- A system's internal loops that convert environmental sensing and modeling into regulation of its own behavior and state, forming a self-sufficient circuit.
- functor
- A category theory tool that maps objects and relations from one mathematical or physical system onto another.
- divisive normalization
- A neural circuit that scales a population's output by dividing by the overall activity level.
- winner-take-all
- A neural circuit where mutual inhibition between populations leaves only one remaining active.
- wave walker
- A droplet bouncing on a vibrating oil surface that leaves ripples feeding back to guide motion, forming a Turing-complete yet externally-closed system.
How to listen
Engineers skeptical of AI agent hype and researchers seeking to clarify whether 'computation' describes brains and AI systems.
The category theory and functor mathematics (around 30:40–31:43) can be skipped without a theory background.