Are Dreams Just Next-Token Prediction?

Rethinking Inspiration and Creativity in the Age of Generative AI

🗓️

Rethinking Inspiration and Creativity in the Age of Generative AI

In your stream, live in a dream. (Red Hot Chili Peppers, I like dirt)

It’s a profound question, isn’t it? When we dream, are we merely running a biological version of generative AI? And if that’s true, what does it mean for inspiration, creativity, and the human spirit?

Imagine, for a moment, how we experience dreams. In dreams, we encounter a blend of the fantastic and the familiar. Faces of people we know in places we’ve never been. Entire worlds that, although they don’t quite make sense, feel intensely real. We even have rules, story arcs, and dialogue. And sometimes, upon waking, we ask, “Where did that come from?” We might wonder if dreams are telling us something deep or if they’re just random.

In today’s AI-powered world, we see machines that also create—at least, in a sense. They take what we’ve fed them, interpret patterns, and produce text, images, sounds, and even videos. They give us outputs based on patterns we recognize, even if we might find them strange or unexpected. So let’s ask: Are dreams just a form of generative AI?

The Nature of Prediction in Both Dreaming and AI

At its core, generative AI is built on a process called “next-token prediction.” It learns from vast amounts of text, images, and data, and then, from a starting point, it predicts the most likely next step in a sequence. And it does this over and over, refining and optimizing for accuracy and fluency.

But our brains, and our dreams, don’t work that way—at least, not exactly. Our dreams are generated from layers of sensory experience, memory, emotion, and interpretation, forming intricate stories. They’re not optimized for the next “logical” step; they’re optimized for something else: meaning.

The Paradox of Inspiration: Between Prediction and Possibility

And this brings us to inspiration. If dreams are our mind’s “generative AI,” then is inspiration merely the quality of “good” next-token prediction? Or is it, perhaps, a break from the norm—a way of drawing us beyond the next logical step to something uniquely resonant?

When AI creates, it’s striving for coherence, fluency, and sometimes an element of surprise. But it’s not truly inspired. It’s generating output that optimizes for probability, not for emotion or new possibility. True inspiration, whether it arises in dreams or creative endeavors, often lives in the unexpected, the gap, the moment of surprise where we see something we haven’t encountered before.

Quality and Efficiency vs. The Value of Novelty

Generative AI is trained to optimize, to produce efficiently and on command, a sequence of data or words or pixels that we, as humans, will recognize as having meaning or structure. But humans don’t dream to produce something useful. In dreams, the bizarre and nonsensical holds as much value as the logical and coherent. Sometimes, it’s in these wild, free-flowing ideas that our best inspiration lies.

Consider this: If a person dreams of flying over a strange city, it’s not because that’s a likely “next token” for their brain to produce. It’s because the experience is driven by the mind’s openness to play, to imagine outside constraints. AI, for all its creative appearance, is inherently bound by its constraints: its training data, its algorithms, and its mathematical structure. Dreams, however, are not bound by logic or structure.

Is Inspiration Just an Optimization Problem?

When we’re inspired, we’re tapping into something generative AI lacks: the ability to connect deeply with emotions, memories, and experiences and then interpret them through an entirely human lens. Inspiration arises from the depths of human experience, shaped by moments, losses, joys, and even the mysteries we hold within ourselves. It’s not about efficiency; it’s about experience and connection.

Generative AI can compose, narrate, and paint, but it’s always limited to the scope of its data. Human creativity, on the other hand, is boundless because we don’t stop at optimizing for the “next token” in life. We aren’t optimizing for the next perfect line; we’re seeking the next profound line—the one that transforms us.

Are We Just Predictors or True Creators?

So, are dreams just next-token prediction? Are we, ourselves, merely machines refining the next best step?

I believe the answer lies in our willingness to embrace the unknowable and to take inspiration from the chaos and mystery of our minds. Dreams and inspiration are not about efficiency or logic; they are about meaning and impact. Generative AI may help us with the brushstrokes of creation, but it is us—the artists, the dreamers—who give the painting its purpose, its beauty, and its truth.

🗓️
⏱️ Last updated on