# How does computing power limit what AI models can do?

Date: 2026-10-18 · Author: Robert Scales · https://guidetoartificialintelligence.com/en/learn/how-compute-limits-models/

**Answer:** Compute limits AI in three places at once, how large a model a lab can train, how many experiments it can run, and how many people it can serve. All three draw on the same chips, so a lab that serves more users has less left for research, which is why you see usage caps, limited launches and cancelled products.

"Compute" is shorthand for the processing capacity a company can use: the chips, plus the power and data centres that keep them running. Unlike a website, an AI model costs real money each time it is used. In a Scientific American interview, the AI policy researcher Lennart Heim explains it simply: using AI 10 times as much costs the provider roughly 10 times as much.

The general case for why compute decides who leads is in [our overview of chips and power](/en/learn/chips-compute-sovereign-ai-canada/). This article looks at the mechanism: where the limit actually bites.

## One pool, three jobs

Think of a lab's chips as one fixed budget of machine hours, split among three jobs.

| Job | What it means | What a shortage does |
|---|---|---|
| Final training runs | Producing the model you eventually use | Smaller or slower models |
| Research and experiments | Trying ideas, most of which fail | Fewer ideas tested, slower progress |
| Inference (serving) | Answering users and running agents | Usage caps, slower answers, higher prices |

Heim suggests thinking of it as R&D compute against serving compute, instead of the more common training against inference. The numbers show why.

## Worked example: where OpenAI's compute went

Epoch AI, a research group that tracks AI resources, estimated OpenAI's 2024 cloud spending from press reports and its own cost models. These are estimates with wide error bars, not company disclosures.

| Category | Estimated 2024 spending | Share of about $7 billion |
|---|---|---|
| GPT-4.5 final training run | About $400 million | About 6% |
| Final runs for other released models | About $80 million | About 1% |
| Other R&D: research, experiments, unreleased models | About $4.5 billion | About 65% |
| Inference | About $1.8 billion | About 26% |

(The shares are our rounding and do not sum exactly.) Epoch reads this as most R&D compute going to experiments, not to the runs that produced released models. The famous "cost to train a model" is therefore a small slice of what it took to get there.

DeepSeek's V3 paper makes the same distinction from the other side. It reports 2,048 H800 GPUs and a $5.576 million official training run, and it says that figure leaves out earlier research and experiments. A lab with fewer chips can still produce a strong final run. What it gives up is the number of attempts.

The mix also moves. Epoch says that OpenAI's compute in 2025 split roughly evenly between R&D and inference, with R&D having taken the higher share the year before. As users arrive, serving claims more of the pool.

## How fast the pool is growing

Compute is scaling quickly, and demand is scaling quicker.

| Measure | Figure |
|---|---|
| OpenAI's stated power capacity, year-end | 0.2 GW in 2023, 0.6 GW in 2024, 1.9 GW in 2025 (approximate) |
| In H100-equivalent chips (Epoch's estimate) | About 1.74 million at OpenAI at the end of 2025 |
| Google DeepMind (Epoch's estimate) | About 1.58 million, with a range of about 1 to 2.5 million |
| Anthropic (Epoch's estimate) | About 1.19 million |
| Meta Superintelligence Labs (Epoch's estimate) | About 1 million |
| SpaceXAI (Epoch's estimate) | About 0.6 million |

An H100-equivalent expresses mixed fleets as the number of Nvidia H100 GPUs that would match them. Epoch notes that OpenAI and Anthropic own very little of this hardware and mostly rent it from cloud providers.

## Where you can see the limits

Limits rarely appear as an announcement. They show up as product decisions:

- **Usage caps.** Scientific American reports that in late March 2026, some heavy Claude users hit five-hour limits in 20 minutes, and that Anthropic told subscribers sessions would run down faster at peak hours. It also blocked some third-party tools from drawing on flat-rate plans. Anthropic is not unusual here, but it is a clear example of what scarcity looks like to a paying customer.
- **Dropped products.** The same article says OpenAI has begun shutting down Sora, its video tool, as use of its Codex coding assistant surged.
- **Limited launches.** Some top models reach only a few customers at first, a pattern covered in [why top models launch limited](/en/learn/why-top-ai-models-launch-limited-groups/).
- **Design choices.** Labs pick architectures that use less computing per token, such as [mixture-of-experts](/en/learn/moe-vs-dense-architecture/), and cheaper "small" models handle routine requests.

Heim's explanation for capacity shortfalls is that overbuilding is very costly and demand is hard to forecast, so companies try to build only what they need and can fall short. In his view, they currently prefer rationing to raising prices, so that everyone keeps some access. Further down the chain, the limits are physical: chip factories, [memory](/en/learn/hbm-memory-shortage-explained/), and electricity. TSMC said in October 2026 that the gap between demand and supply remained large.

A caveat on all of this: more compute does not automatically mean a better model. Whether progress keeps scaling with compute is debated, and a clever method can do more with less. The evidence here shows compute limits how many attempts and how many users a lab can handle. It does not show that compute alone decides who wins.

> **Try it:** Note the time of day when your AI tool feels slowest or hits limits first, and try the same task early in the morning for a week. If the limits move with the clock, you are seeing peak-hour demand on shared capacity.

## What can you do when compute limits affect you?

Treat your usage like a budget, because it is one. Long conversations, large file uploads and heavy thinking modes all use more compute per answer. Starting a fresh chat for a new task, sending only the relevant part of a document, and using a smaller model for routine work will usually stretch a plan further.

If your work depends on AI, avoid relying on a single provider at a single time of day. Keep a second tool ready and move non-urgent jobs outside peak hours. Developers can use batch modes where available, cache repeated context, and handle rate-limit errors with retries.

If the work is sensitive or steady, running a smaller open model on your own hardware removes the shared queue entirely, though it adds its own costs and limits. Whichever route you take, assume limits and prices can change, and check your provider's current terms before building a process around them.

## Sources

- Introducing the AI Chip Users explorer: https://epoch.ai/latest/introducing-the-ai-chip-users-explorer
- Most of OpenAI's 2024 compute went to experiments: https://epoch.ai/data-insights/openai-compute-spend
- What is the AI compute crunch, and why are AI tools hitting usage limits?: https://www.scientificamerican.com/article/what-is-the-ai-compute-crunch-and-why-are-ai-tools-hitting-usage-limits/

---

From the Guide to Artificial Intelligence, by Robert Scales, published by Holistic Hybrid Inc. Share and adapt it under CC BY-NC-SA 4.0 (https://creativecommons.org/licenses/by-nc-sa/4.0/): credit the guide, link to https://guidetoartificialintelligence.com/en/learn/how-compute-limits-models/, no commercial use without permission, and keep the same licence.