Teaching AIs to Imagine & Reason

Why do today’s chatbots sometimes seem brilliant and then suddenly feel limited? A key reason is how they’re trained: most still predict the next word—technically, the next token—one at a time. This left-to-right approach works for finishing emails, but it restricts the bold leaps of imagination and multi-step reasoning we desire.

In this episode, we explore two groundbreaking studies that challenge these norms:

1.

See the entire picture at once. New systems treat every token as part of a bigger scene, like sketching out a painting’s outline before adding colors. This broader view helps AI recognize long-range patterns and decreases the chances of simply copying answers.

2.

Choose your process. Another technique allows the model to decide which blank to fill first—and second, and third—while generating. On a Sudoku-style test, this simple freedom boosted accuracy from 7 percent to nearly 90 percent, even outperforming larger, more expensive models stuck in strict left-to-right mode.

We also highlight a surprisingly simple hack called seed-conditioning. By inserting a few random characters into the input—the “seed”—researchers encourage the AI to produce more surprising, original responses without adjusting any temperature settings.

Together, these ideas suggest a future where AI models do more than just autocomplete—they plan, improvise, and create. Whether you're developing enterprise tools or curious how machines might write the next best novel, this discussion will challenge your ideas about what AI can do beyond predicting the next token.

This episode is based on the research papers “Roll the Dice & Look Before You Leap: Going Beyond the Creative Limits of Next-Token Prediction” (Nagarajan et al., 2025) and “Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions” (Kim et al., 2025).