Intelligence Without Memory: The Billion-Parameter Bet

Andrej Karpathy, in a series of talks and writings through late 2025, articulated what may be the most important unsolved problem in AI: these systems have no memory. Not in the colloquial sense of “they forget the beginning of a long conversation.” In the deep architectural sense that nothing that happens during inference changes the model. Every conversation starts from zero. Every user is a stranger. Every lesson is forgotten the moment the context window closes.

This essay explores why that matters, what it implies, and where it leads.

Ghosts in the Machine

Consider what happens when you use a language model. You provide context. The model generates a response. You refine your prompt. The model generates a better response. Over the course of a conversation, you develop a shared understanding. The model seems to learn your preferences, your vocabulary, your level of expertise.

Then you close the tab. Everything vanishes.

The next conversation starts from the same weights. The model has not changed. It has no record of your interaction. The “learning” that appeared to happen was an illusion created by the context window: a temporary scratch pad that mimics memory but is not memory.

Karpathy’s insight is that this is not a minor limitation. It is the defining architectural constraint of current AI systems. Human intelligence is inseparable from memory. We learn from experience. We adapt to individuals. We build on past interactions. We develop expertise over time. An intelligence without memory is not a lesser intelligence. It is a fundamentally different kind of thing.

We have built systems with the reasoning capacity of a doctor who has amnesia: brilliant in the moment, incapable of growth.

Sucking Supervision Through a Straw

Karpathy used a vivid metaphor to describe how current AI systems learn: they suck all their supervision through the narrow straw of the training process. Everything the model knows must be encoded in its parameters during training. After training, the model is frozen. It can process new inputs, but it cannot update its understanding.

This creates a profound bottleneck. The world changes continuously. The model’s knowledge is fixed at a point in time. Fine-tuning and retrieval-augmented generation (RAG) are patches, not solutions. Fine-tuning is expensive and infrequent. RAG adds information to the context but does not change the model’s understanding.

The metaphor extends further. Human learning is not a one-time download followed by read-only access. We learn continuously, from every interaction, adjusting our models of the world in real time. We do not need to be “retrained” to incorporate new information. We simply update. Current AI architectures have no equivalent mechanism.

The implications for product design are immediate:

• Personalisation is fake. Without persistent memory, any personalisation must be reconstructed from scratch each session. User preferences, communication styles, domain-specific knowledge, all lost.
• Expertise cannot accumulate. A medical AI that has seen ten million patient interactions is no better at the ten-million-and-first than it was at the first, unless explicitly retrained.
• Trust cannot compound. Trust is built through repeated successful interactions. A system that forgets every interaction cannot build trust through experience.

The Demo-to-Product Gap

Karpathy identified a pattern that explains much of the disillusionment with AI products: demos showcase the model’s peak capability in a single interaction. Products require sustained capability across thousands of interactions. The gap between these two is the memory gap.

A demo can be carefully prompted. The context can be curated. The interaction can be guided. A product must handle whatever the user throws at it, in whatever order, across whatever time horizon. Without memory, the product cannot learn from its mistakes, cannot adapt to its users, cannot improve with use.

This explains the paradox of AI products that demo brilliantly and disappoint in production. The demo tests intelligence. The product tests intelligence plus memory plus adaptation. Current systems have only the first.

The result is a specific shape of user frustration: “It was amazing the first time. Then it kept making the same mistake.” The system is not degrading. It is simply unable to learn from the correction. Every session is the first session.

The Cognitive Core

Karpathy proposed a way to think about what language models actually are: cognitive cores. Pure reasoning engines without the memory, learning, and adaptation systems that biological intelligence wraps around its cognitive core.

A human brain has a cognitive core too: the ability to reason, to draw inferences, to solve problems. But that core is embedded in systems that provide:

• Episodic memory: recall of specific past experiences
• Semantic memory: accumulated factual knowledge that updates continuously
• Procedural memory: learned skills that improve with practice
• Working memory: the ability to hold and manipulate information across tasks
• Metacognition: the ability to monitor and adjust one’s own reasoning

Current LLMs have a cognitive core and a rudimentary working memory (the context window). They lack everything else. The research agenda, in Karpathy’s view, is not to make the cognitive core better. It is to build the memory and adaptation systems around it.

This framing is liberating. It means the current limitations of AI are not evidence that the approach is wrong. They are evidence that the approach is incomplete. The cognitive core works. What we lack is the surrounding architecture.

Education as Empowerment

Karpathy’s most practically important argument concerned education. If AI systems cannot accumulate expertise through experience, then the humans who use them must be expert enough to guide them. The model is a tool, and like all tools, its output depends on the skill of the operator.

This inverts the common narrative that AI will replace the need for human expertise. Karpathy argued the opposite: AI increases the return on human expertise. An expert using an AI tool produces far better results than a novice using the same tool, precisely because the expert knows what to ask for, how to evaluate the output, and when the model is wrong.

The practical implications:

• AI literacy is not optional. Understanding what models can and cannot do is a professional skill as fundamental as numeracy or written communication.
• Domain expertise becomes more valuable, not less. The model provides the cognitive power. The human provides the direction, the judgment, and the memory of what has worked before.
• The best AI products will be educator products. They will not hide the model’s limitations. They will teach users to work within and around them.

Karpathy’s vision is ultimately optimistic, but it is an optimism grounded in a clear-eyed assessment of current limitations. The billion-parameter bet is that cognitive cores will eventually be wrapped in the memory, learning, and adaptation systems they need. But until that happens, the systems we have are brilliant amnesiacs: capable of extraordinary reasoning, incapable of remembering what they reasoned about yesterday.

Intelligence without memory is not intelligence in any sense that matters for sustained value creation. It is computation. Impressive, useful, profitable computation. But until these systems can learn from experience, the human in the loop is not just advisable. It is architecturally necessary.