A practical breakdown of the AI power situation

I’m in my 30s now, so it won’t surprise you to hear that I’ve been listening to a bunch of Sarah Paine lectures about World War II recently. One of the themes I keep coming back to is the major (even primary) role that resources play in large scale conflicts: the U.S. industrial output (and to a lesser extent, The Allies’) was simply far superior to The Axis and that’s a big reason that we won. It’s not just the weapons and the soldiers that fight wars; it’s all of the enabling infrastructure that supports them.

I mention this because the same exact thing is true in AI, which some (not me) are starting to talk about as a type of war itself. Much of the focus in AI is on state of the art model capabilities and whatever the newest frontier model can do. But equally important is all of the infrastructure that goes into creating and running those models.

The market knows this, because it thinks NVIDIA is the most valuable company in the world. And it seems like every day OpenAI is signing yet another obscure deal with a cloud provider, chip maker, or data center buildout firm (and many other wacky characters too).

The goal of this post is to give a gentle, practical introduction to what’s going on with AI, data centers, and power; to try to give a little insight into these mega deals that don’t seem to make much sense.

I don’t think it’s controversial at all to say that the biggest bottleneck to the future of AI isn't the code, and it isn't the chips (though that’s part of it). It’s physics, or the ability to pull electrons out of a wall socket.

The Scaling Law: more compute = better AI models

The first step towards understanding our current situation is the scaling law. We’ve written about this extensively before. The basic gist is that over the long run, the reason that AI models get better and better is that we are training (and using) them with more, bigger, powerful computers. For some reason – nobody is sure why – the more compute you throw at the problem the better models tend to get.

When it comes to AI what scaling up compute typically means is just more GPUs. GPUs are built for parallel computing. Instead of having a task done all by one worker, breaking it down into different parts and having multiple workers work on it at once. This is a perfect fit for how you train and run an AI model, which is in essence “just a bunch of matrix multiplication.” So as far as the Scaling Law is concerned, the more GPUs you can get and run, the better your models will get.

Right now, the leaders at OpenAI, Anthropic, and Google are looking at their charts and seeing that the line for model performance keeps going up. They haven't hit a wall yet. They believe that if they can just build a cluster of 100,000 GPUs (or some absurd number like this) they might reach Artificial General Intelligence (AGI).

The catch is that GPUs are unbelievably power hungry. A rack of standard servers in a data center a decade ago might have consumed 4 to 6 kilowatts of power. A rack of NVIDIA’s newest Blackwell chips can consume upwards of 100 kilowatts. The reason for this is that architecture difference I mentioned earlier: unlike CPUs, GPUs are doing tons and tons of parallel tasks, all of which need to be powered at the same time.

So where is all of this power going to come from?

The Grid: A 20th Century Solution to a 21st Century Problem

Some people will tell you that a lot of AI lab strategy comes down to game theory (again with the war, Justin). If you take the realities of model development to their logical conclusions, this means that AI labs will essentially want an infinite amount of compute, which means they will need an infinite amount of power. The problem is that the United States power grid is... well, it’s a bit of a mess.

For the last 15 years, power demand in the US has been pretty flat. We got better at efficiency (LED lightbulbs, energy-star fridges), so even though we are using more electronics, the total electricity usage didn't jump much.

This is a good thing, to be clear: it’s nice when we have enough energy to support ourselves. But it had a pernicious long term effect. Because demand was flat, utility companies didn't build a ton of new power plants. The goal was just to maintain what we already have. Then, BOOM. AI happens, and data centers are suddenly requesting as much power as the city of Miami.

This narrative is true outside of the U.S., too. Total electricity consumption in Europe has stayed remarkably steady over the past few decades. And if you’re on Twitter as much as I am, you’re probably aware that many have been criticizing their attitude towards Nuclear…but that’s none of my business.

The point I’m trying to make here is that if we’re going to radically scale up the number of GPUs we’re using, something drastic is going to have to change in the grid. If AI labs have their way, within a few years GPUs in data centers are going to be consuming more power – way more power – than all other sources of energy consumption combined. We can’t just give rig workers more overtime, we need a complete revolution.

Compounding this problem is that typically, building new transmission lines in the U.S. takes a really long time. Usually years, sometimes even a decade. There are just layers and layers of permits, regulations, and people who don’t want ugly wires in their backyards. AI labs obviously cannot wait years to break ground on a new data center. They need power yesterday.

The three way handshake and how these deals are structured

To put some numbers to this, the typical cloud data center – which is a massive, power hungry, technological marvel by any measure – consumes anywhere from 0.1GW to 1GW. On the larger end of this spectrum, we are talking about multiple physical buildings aggregated together (a campus, so to say)[1] By the way, 1GW of capacity usually costs $50-60 BILLION to build. . But we max out at 1GW, yes? OpenAI’s deal with NVIDIA plans for 10GW of capacity. Ten! And that’s just one of several deals that OpenAI has signed. If you add them all up, OpenAI is planning on >25GW of capacity in the near future 🤯.

Now, I do think it’s important to exercise some skepticism here. I’m no Ed Zitron, but I don’t take any of these planned GW capacity numbers at face value. We are clearly in some sort of bubble, and these deals have important political benefits for AI labs beyond just the literal power buildout. But even if you want to discount the numbers by half, or even 90%, we are still talking about a buildout of unprecedented scale and ambition.

Because the grid is way too slow to match even a small portion of these insane ambitions, the labs are taking matters into their own hands. We are seeing a massive consolidation of power, literally and metaphorically. This has created a new "three-way handshake" (which my sources tell me is physically doable) that defines the AI economy:

1. The brain: OpenAI, Anthropic, Google DeepMind. They design and build the models.
2. The muscle (cloud + chips): Microsoft Azure, AWS, NVIDIA, Broadcom. They provide the infrastructure.
3. The fuel (energy): Constellation Energy, Talen, Dominion. They provide the electricity.

So what are these deals, exactly? What does it actually mean when you see a headline like “OpenAI and NVIDIA Announce Strategic Partnership to Deploy 10 Gigawatts of NVIDIA Systems?” Well first, they’re all idiosyncratic. But generally you can think of the arrangement as a kind of investment. For the OpenAI <> NVIDIA deal, NVIDIA agreed to invest $100B into OpenAI, and in return OpenAI will use a metric fuckton of NVIDIA chips and also give NVIDIA more equity.

Or take Anthropic’s group deal with NVIDIA and Microsoft. NVIDIA is investing $10B and Microsoft is investing $5B, and outside of that the details are pretty scant. But we also know that Anthropic is somehow investing $50B into building data centers with Fluidstack in Texas and New York. Where will the power for these come from?

Now, despite the fact that the headlines will quote a power number – e.g. 10GW – this does not mean that OpenAI or NVIDIA are getting into the energy business. They are outsourcing that hairy part of the equation to power companies, and it won’t surprise you that there is scant information on the web about who these suppliers actually are.

One thing we do know is that Nuclear is going to have to be an option here. In 2024, Microsoft signed a deal to re-open the Three Mile Island plant in Pennsylvania, which will produce just under 1GW of power when up and running. Constellation Energy is managing the buildout, and I think it’s safe to assume they’ll play a major role in some of these other deals that we’re seeing today. Amazon also bought a data center campus right next to a nuclear plant in Pennsylvania so they could plug directly into the reactor (literally "behind the meter").

It’s important – very important – to remember that energy is a stack. There are many, many different hardware components that go into “simply procuring” huge amounts of electricity: things like turbines, cables, etc., and many of the companies that manufacture these things are completely booked out past 2028. Nobody does a better job of putting together the numbers here than Dylan.

By the way, if you want to see a concise summary of what I said earlier about the scaling law and power, look no further than Mr. Sam himself:

“Everything starts with compute. Compute infrastructure will be the basis for the economy of the future, and we will utilize what we’re building with NVIDIA to both create new AI breakthroughs and empower people and businesses with them at scale."

I am not exaggerating. Altman is singularly focused on securing compute and ergo securing power. In his mind it is all that matters on a long enough timeline.

Recap

• The Scaling Law rules everything: The smartest people in the room believe that if we just throw more computers at AI, it keeps getting smarter. There is no ceiling yet.
• More compute means more data centers: scaling up compute means running more, larger CPUs and GPUs in more, larger data centers across the world.
• But the grid can’t support these DCs: The U.S. power grid was built for a world of lightbulbs and TVs, not for millions of H100 GPUs running 24/7. We are running out of supply.
• The nuclear option: Big Tech (Microsoft, Google, Amazon) is literally buying nuclear power plants because wind and solar aren't reliable enough to keep the AI brain alive.
• The supply chain of intelligence: Everyone who matters in AI is signing huge contracts to build out more data centers and power: AI labs (OpenAI), cloud providers (Microsoft/Azure), and energy companies.ing.