Imagine you took 150,000 completely different cars — different makes, different engines, different drivers — pointed them all at the same destination, and let them go. You’d expect them to fan out across every road in the country, right?

That’s not what happened.

In a 2024 study published in the Proceedings of the National Academy of Sciences, researchers Jialin Mao, Itay Griniasty, Han Kheng Teoh, Rahul Ramesh, Rubing Yang, Mark K. Transtrum, James P. Sethna, and Pratik Chaudhari did something similar with AI. They trained over 150,000 versions of AI models — tiny ones, massive ones, models trained with completely different methods, different architectures, different settings — and tracked the learning journey of each one.

They expected chaos. They found something much stranger.

Every AI Takes the Same Road

To understand what they found, picture this. At the start of training, an AI model knows nothing. It’s basically guessing. Call that point “ignorance.” At the end of training, it’s learned the task. Call that “truth.” The question is: what path does each model take between those two points?

You could think of all possible AI answers as a vast, high-dimensional space — like a universe of predictions. Each model, at each moment during training, sits at one point in that universe. As it learns, it moves.

The researchers expected these paths to scatter everywhere. With so many different setups, why would any two models take the same route?

But they didn’t scatter. They funnelled together.

Despite enormous differences in architecture, training method, model size, and settings, all the models followed almost identical paths. Imagine a ribbon stretched from ignorance to truth — narrow, surprisingly direct — and nearly every model stayed on it.

Three dimensions were enough to describe 76% of the differences between all 150,000+ models. A space that should have been impossibly complex turned out to be, in practical terms, almost flat.

What Doesn’t Change the Path

Here’s the part that surprised even the researchers: most of the things developers spend time tweaking barely changed the route at all.

Different optimisation algorithms. Different batch sizes. Different regularisation techniques. Different weight initialisations. None of it shifted the path in any meaningful way.

The one thing that did matter? Architecture — the fundamental shape of the model itself. A convolutional network took a slightly different route than a transformer, for instance. But within each architectural family, variations in how you train made almost no difference to the path.

And bigger models? They didn’t learn differently. They just moved faster along the same ribbon. A larger model made more progress per step. The road was the same.

A Map We Didn’t Know Existed

This is a remarkable finding. We tend to think of AI training as unpredictable — a complex process that produces different results based on thousands of choices. And in some ways, it is. But underneath that complexity, something consistent is happening.

Learning, it seems, follows a hidden map.

Why? The researchers believe it comes down to the structure of typical data. Real-world information — images, text, sounds — tends to have a kind of natural hierarchy. A few patterns matter a lot; many patterns matter very little. That hierarchy shapes the path an AI takes when it learns, pulling different models toward the same corridor regardless of how they were built.

It’s not that every AI is the same. But the process of learning from real-world data appears to be far more constrained than anyone expected.

---

This article is based on research by Jialin Mao, Itay Griniasty, Han Kheng Teoh, Rahul Ramesh, Rubing Yang, Mark K. Transtrum, James P. Sethna, and Pratik Chaudhari. The full dissertation, “The Hidden Geometry of Learning” (University of Pennsylvania, 2025), is available at pratikac.github.io/pub/mao.thesis25.pdf. The key findings were published in the Proceedings of the National Academy of Sciences (2024).