The AI Grid Squeeze: Why Your Cloud AI Bill Isn't Falling

Your AI bill went up this year, and every price you're paying went down. That isn't a billing error. It's the central structural fact about building AI products right now, and it breaks the budget math most teams wrote in 2024: tokens got dramatically cheaper while the physical system that produces them started to bind in a way chips never did.

Tokens got cheaper. Your invoice didn't.

Anthropic's Claude Opus 4.5, in November 2025, cut frontier pricing from $15/$75 per million input/output tokens to $5/$25, roughly two-thirds off the top tier in one release. Google's Gemini 3.1 Pro sits at $2/$12 per million. Workhorse models are cheaper still: Gemini 3.1 Flash-Lite at $0.25/$1.50, Claude Haiku 4.5 at $1/$5.

Epoch AI found the price of hitting a fixed benchmark score falling 9× to 900× per year depending on the task, median around 50×; matching GPT-4 on PhD-level science questions got roughly 40× cheaper per year. Freeze your 2024 capability requirements and serving them today would be nearly free. Nobody freezes their requirements.

The unit got 40× cheaper and you bought 400× more units. That's not a pricing failure — that's Jevons paradox with a login screen.

Agentic workflows are the mechanism. A chatbot turn is one prompt and one completion. An agent is stateful: tool outputs, reasoning and prior turns accumulate, and that history rides the context window on every single inference call. A twenty-step trace can carry 50,000 tokens of context re-billed at each step. Cheaper capability also makes uneconomic features viable, so savings get spent on scope. For the mechanics of tokens and caching tiers, see how AI pricing really works.

The bottleneck moved from silicon to electrons

For most of cloud history the scarce thing was the chip. In 2026 it's the power for the rack, and whether you can plug it in at all.

• Goldman Sachs Research projects U.S. data center power demand climbing from about 31 GW in 2025 to roughly 41 GW in 2026 — 4.1 % to 5.3 % of U.S. peak summer demand. EPRI's scenarios put them at 9–17 % of U.S. electricity by 2030, up from roughly 4.5 % today.
• PJM Interconnection, the largest U.S. grid operator, held its 2028/2029 capacity auction on July 14, 2026 and came up 6,831 MW short of its reliability requirement — the second consecutive year the entire RTO has fallen short. It cleared at the $325/MW-day price cap, with PJM's peak load forecast up roughly 2,000 MW year over year, largely from data centers.
• Interconnection queues are the real gate: about 25 months nationally on average, 36–48 months for PJM projects in data center load-growth zones, and four to seven years in Northern Virginia, Phoenix and Dallas.
• Even the silicon story is a supply story. TSMC's CoWoS advanced packaging, which bonds HBM stacks onto the GPU substrate, is allocated through at least mid-2027, with data center GPU lead times of roughly 36 to 52 weeks for non-priority buyers.

The surprise isn't that GPUs are unobtainable. It's that even when they exist, the substation doesn't.

Capacity is consumed the moment it's added. Goldman expects U.S. data center capacity to roughly double to about 95 GW by the end of 2027, and demand forecasts keep rising to meet it. No slack means no pressure to cut prices.

The operators are becoming energy companies. By mid-2026 every major hyperscaler had signed nuclear supply, roughly 9.8 GW committed across a dozen-plus disclosed projects: Microsoft's 20-year PPA with Constellation to restart Three Mile Island Unit 1 (835 MW, first power targeted for 2027), Google's commitment to up to 500 MW of Kairos Power small modular reactors (first unit around 2030), and Amazon's Talen offtake expanding toward 2 GW plus the $700M round it led in X-energy. Long-payback capital programs do not produce cheap tokens next quarter.

One myth worth killing: the per-query energy panic

The claim that one AI query burns ten — or a hundred — times the electricity of a web search was a reasonable guess in 2023. The measurements say otherwise. Google published real numbers on Gemini: a median text prompt uses about 0.24 watt-hours, roughly nine seconds of television, and it cut energy per prompt 33× over twelve months. Microsoft puts a typical production LLM query at 0.16 to 0.60 Wh, median near 0.31 Wh — a web search is in the same neighborhood. Long reasoning runs land in the tens of watt-hours, but the casual prompt is not the villain.

What actually got cheaper — and what didn't

"AI prices aren't falling" is too coarse, and so is "AI is getting cheap." The picture splits by line item.

The cost levers that actually work

The levers on the product side got stronger, not weaker, roughly by return on effort:

1. Model routing. Send routine queries to a small model; escalate to the frontier tier only when a classifier says the query needs it. With Flash-Lite class models an order of magnitude below frontier pricing, that's a multiple, not a percentage. Our honest comparison of the 2026 model lineup covers which tier earns its price.
2. Prompt caching. The antidote to the agentic token explosion: a repeated system prompt or tool schema becomes a near-free lookup instead of a full re-encode every step — often the biggest single line-item change available.
3. Context hygiene in agent loops. Summarize or prune history instead of dragging the full trace through step twenty; most frameworks default to the expensive behavior.
4. Streaming, early termination and batching. Stop generation once the answer is complete; push offline enrichment onto batch APIs.
5. Evaluation before "upgrading." Prove on real test cases that the frontier model is materially better before routing traffic to it. Half of "we need the biggest model" doesn't survive an eval.

Notice what's not on that list: "wait for prices to fall." They already fell, and it didn't help, because nothing about waiting touches volume.

Where people go wrong (and when to call a pro)

Frequently asked questions