Score AI infrastructure: power, GPUs, network, $/token, p95

Power is the first price

On the vendor slide, GPU count is the headline. The cost driver is contracted power. The trade-off is arithmetic: power divided by scheduled tokens sets the cost floor, while network distance divided by the p95 budget sets the latency tax. GPU count alone tells you neither.

A central scenario puts cumulative data center capital spending at $31.6 trillion through 2050. Annual spending is projected at about $800 billion in 2026, $1.1 trillion in 2030, and $1.8 trillion in 2050. The 2025 computing and AI market was about $215 billion. Treat those macro numbers as context, not as a vendor's price list.

Construction is the smaller commitment. One dollar of construction spending can lead to about twelve dollars of later equipment investment. Information and communications technology equipment is projected to be about 70% of data center capital in 2026 and 93% by 2050.

Ask for the contracted power envelope, the utility interconnection date, and the backup plan. A site can have a large building and still be limited by the grid. A missing interconnection date means the capacity is unproven. Check the cooling budget, because heat limits sustained token rate.

GPU count is a utilization problem

Raw GPU count is a marketing number. Request scheduled utilization, not rack count. Large GPU pools with only part reserved for your workload shrink your effective capacity. That changes cost per token before the first token ships.

Data center capital distribution depends on electricity, latency and connectivity, trusted-region security, GPU access and ecosystem depth, and policy plus community consent. GPU data centers can look full in a pitch and still be underused in production. Reserved capacity, not installed capacity, is the metric to compare.

Obtain the reservation model. Some vendors sell shared capacity; others sell dedicated capacity. Shared capacity can be cheaper, but it can also be preempted. Know whether the GPUs are reserved, burstable, or best-effort.

Billed GPUs that sit idle still consume power and network. Subtract idle time from the billed GPU-hours before computing cost per token. Compare the reservation to the model's batch size and concurrency.

Network distance is a p95 tax

Latency is not a footnote. About 30% of workloads already carry localized requirements, and that share is growing. For inference, a slow path can raise p95 even when average throughput looks fine. Measure round-trip time to the serving region, not just bandwidth.

Trusted-region security affects where tokens can be served. For users in a regulated region, a distant cluster may fail the latency budget before it fails the security review. Keep the serving region, model, and user geography in one document.

Get the serving region, the peering path, and the measured p95 from your user base. A provider's internal dashboard may show a good number, but your users may sit behind a different route. Run the test from the same geography that will pay for the tokens. Record the test date, region, and model size.

Network distance also affects retries. When p95 misses the budget, clients may retry, which raises token volume and cost. Include retry overhead in the $/token model.

Run this scorecard before signature

Apply the same buyer scorecard whether the vendor is a data center, a GPU cloud, or an inference provider. The five scorecard questions below run from physical limit to financial penalty. Complete each item before signing.

• Power availability: request contracted MW and backup; if the number is missing, stop.
• GPU count and utilization: pull reserved GPU-hours and scheduled utilization; match them to your token target.
• Network/latency: measure p95 round-trip from your users to the serving region; test it against your latency budget.
• $/token under load: require a load test at peak token rate, not an idle quote.
• p95/outage cost: add service-level credits and an outage cost formula tied to p95 misses.