Use cases · Reference architectures
USDC plans deployments in a repeatable way. Each reference architecture runs the same seven blocks in the same order, states only sourced figures and closes with what it does not solve.
Most buyers need two to three megawatts now and cannot commit to twenty. The pod is the unit of purchase, so the second and sixth pod land on the same design as the first.
Serving a model has two phases with opposite hardware appetites. Splitting them inside one pod lets each phase run on the silicon it actually needs.
Agentic workloads send the same long context back to the model over and over. A shared cache tier turns that repetition from a cost into an advantage and it only works if the sites sit on good fiber.
Bring the near-term number you can defend and the interconnect date you are working toward. USDC will plan the campus around both.
Let's plan your campus
with confidence.
Vera Rubin NVL72 Facility Planning Summary (NVIDIA) • DigiPowerX Cerebras CS4 Business Case.
Reference IT loads are published planning figures, not USDC measured results. Campus pod counts are illustrative.
Most buyers need two to three megawatts now and cannot commit to twenty. The pod is the unit of purchase, so the second and sixth pod land on the same design as the first. USDC prepares the shared campus once; compute is added in pod increments as demand arrives.
An AI company has a funded workload for the next twelve months and a demand curve after that which nobody can forecast honestly. It needs two to three megawatts of capacity now.
Funded near-term workload. The number the company can defend today.
Demand curve uncertain. A build-to-suit asks for a shell sized for the whole curve.
A build-to-suit data center asks the company to commit to a shell sized for the whole curve, then wait eighteen to twenty four months to occupy it.
The company signs for what it can defend today, but the facility it signed into was designed once—around one power topology, one cooling loop and one rack density.
02 / 07
The Constraint
Modularity is usually sold as a speed argument. Speed is real, but it is not the important part. A modular pod makes the engineering decision repeatable: if the pod is the unit of design, the tenth pod is the same engineering as the first and adding capacity stops being a redesign.
Traditional expansionDesigned once
Repeatable pod architectureDesigned per pod
03 / 07
What USDC Deploys
One utility interconnect, one perimeter, one cooling plant with headers sized for the end state, one control plane. Only the pods that are needed are built and energized.
Prepare the site for the campus. Build only the pods required today.
Trace a system
Hover or press a system to trace its path through the campus.
04 / 07
How It Works
The shared site work is carried by USDC, not by the tenant. Each phase adds only the capital for what is added.
Phase 01. One pod is built and energized. The substation, pad, yard and cooling headers are already sized for the full campus, so nothing is re-engineered later.
05 / 07
Both figures come from named engineering documents, not from a marketing estimate. They define the envelope a pod must accommodate.
14 IT racks plus 2 network and storage racks in a single row, network racks in the middle.
Vera Rubin NVL72 Facility Planning Summary
11 containers, about 3,970 sq ft.
DigiPowerX Cerebras CS4 Business Case
The argument that separates a repeatable pod campus from a hall built once.
Capacity is added in pod increments rather than hall increments, so capital tracks demand instead of leading it.
The power and cooling design at pod six is the design at pod one. Nothing is re-engineered mid growth.
Silicon is chosen per pod. A later pod can hold a different accelerator generation, or a different vendor, without disturbing the pods already running.
If the demand curve flattens, there is no stranded shell. The pods that were never built were never paid for.
Modular compute does not eliminate
site-level constraints.
The shared elements have to be sized for the end state on day one. Substation capacity, water, land and the utility interconnect are not modular and interconnect queues are measured in quarters or years depending on the ISO. Pod modularity removes the compute commitment risk. It does not remove the interconnect lead time.
Start the campus conversation
with the interconnect date,
not the pod schedule.
Bring the near-term number you can defend and the interconnect date you are working toward. USDC will plan the campus around both.
Let's plan your campus
with confidence.
Vera Rubin NVL72 Facility Planning Summary (NVIDIA) • DigiPowerX Cerebras CS4 Business Case.
Reference IT loads are published planning figures, not USDC measured results. Campus pod counts are illustrative.
Serving a model has two phases with opposite hardware appetites. Splitting them inside one pod lets each phase run on the silicon it actually needs and lets power and cooling be provisioned per role.
The obvious answer is to serve inference on it. The less obvious problem is that inference is not one workload and a pod configured as a uniform block of identical GPUs is the wrong shape for it.
There are three possible outcomes for a training pod. Click any card to inspect.
Homogeneous GPU block
• Fully usedIdle capital
• 0% cluster utilizationWrong shape for
two phases
A homogeneous fleet sized correctly for one of those phases is sized incorrectly for the other. That is true of the silicon and it is equally true of the power and cooling design wrapped around it.
03 / 07
What USDC Deploys
The pod is configured as a decode floor. Alongside it, inside the same pod envelope, a prefill sidecar holds a smaller number of accelerators chosen for raw compute rather than memory bandwidth. The two are joined by the pod fabric and presented to the customer as one pod with two roles.
One pod. Two roles. Power and cooling provisioned per role.
Trace a role
Hover or press a role to see what it owns inside the pod.
04 / 07
How It Works
The serving stack routes an incoming request to a prefill worker, which computes the KV cache and then transfers that cache to a decode worker which produces the output tokens.
The serving stack routes the incoming request to a prefill worker.
The prefill engine reads the prompt and builds the cache under sustained compute load.
In NVIDIA Dynamo the transfer is handled by NIXL, directly from the video memory of the prefill engine to the video memory of the decode engine. The transfer is non blocking, so GPU forward passes continue serving other requests while it happens.
With the SGLang backend, prefill runs as a background task and decode begins immediately while the transfer proceeds in parallel.
The published characteristics USDC plans each role against.
No throughput or latency figures are claimed for
this configuration.
A pod configured as a uniform block of identical GPUs is the wrong shape for inference across two distinct phases.
This is where it becomes a USDC argument.
Prefill sits near sustained TDP while decode is bursty. Metering and cooling them as a single averaged load oversizes one and starves the other. A sidecar lets power and cooling be provisioned per role.
Prefill silicon can be replaced on a different schedule than decode silicon and only the sidecar is opened. The decode floor keeps running.
The two roles do not have to come from the same vendor. A high arithmetic density accelerator can serve prefill while current generation GPUs serve decode. This is where GPU agnostic stops being a slogan.
The same pod serves training between contracts and disaggregated inference under them. The reconfiguration is a software and sidecar change, not a rebuild.
Disaggregation is not always faster.
Disaggregation pays off when prompts are long enough that the cache transfer costs less than recomputing prefill.
For short prompts it can cost more than it saves.
Production stacks handle this by deciding per request whether to disaggregate and the pod supports both modes.
Anyone who says disaggregation is always faster is selling rather than engineering.
Recompute is cheaper.
Disaggregation may cost more.
Transfer is cheaper.
Disaggregation pays off.
Bring the model, the prompt profile and the concurrency you expect. USDC will size the sidecar and the floor around them.
Let's plan your campus
with confidence.
NVIDIA Dynamo documentation, disaggregated serving design notes (NIXL, SGLang backend). No throughput, latency or power figures are claimed for this configuration.
Agentic workloads send the same long context back to the model over and over. A shared cache tier turns that repetition from a cost into an advantage and it only works if the sites sit on good fiber.
Agentic and long context workloads resend the same prompt prefix on every turn. Each repeat that lands on a node without the cache pays the full prefill cost again.
The user sees it as time to first token and the operator sees it as GPU hours spent recomputing something that was already computed an hour ago.
Repeated prefix without cache locality = repeated full prefill cost.
Bad for latency. Bad for efficiency.
Cache held in GPU memory is local, small and lost when the instance moves. Once a deployment grows past a single node the hit rate falls, because the router cannot reliably send a request back to the machine that holds its prefix.
Local GPU cache does not scale. A shared, layered cache tier preserves hit rate and reliability as you grow.
03 / 07
What USDC Deploys
The tier is layered. GPU memory first, then CPU memory, then local NVMe, then a pool that any pod on the site can read. Across the USDC backbone, that pool extends to other sites in the footprint.
Within a site, cache moves at fabric speed. Between sites, at backbone speed.
Trace a scope
Hover or press a scope to see which tiers it spans.
04 / 07
How It Works
Named software, doing the job today.
Runs a cluster wide distributed KV cache pool, with a master server holding metadata and clients on each GPU node.
Moves cache without consuming GPU streaming multiprocessors and without a CPU staging buffer, on dedicated background threads so GPU kernel launches are not blocked.
Does the same job across GPU memory, CPU memory, local SSD and remote backends and reuses cache across requests, sessions and engine instances.
Step 01. Mooncake Store keeps a cluster-wide pool: one master server for metadata, a client on every GPU node.
The published results on agentic traces are the
reason this matters. They describe what the
software layer achieves.
Cache hit rate
Codex traces • vLLM +
Mooncake
Throughput
improvement
Codex traces • vLLM +
Mooncake
Median time to
first token, lower
Codex traces • vLLM +
Mooncake
End-to-end latency,
lower
Codex traces • vLLM +
Mooncake
Near-linear throughput
scaling to 60 GB200 GPUs,
hit rate held above 95%
Codex traces • vLLM +
Mooncake
These are published benchmark figures from the vLLM and Mooncake teams, measured on their traces and their hardware. They describe what the software layer achieves. USDC cites them to explain why the facility design matters. They are not a USDC measured result.
A shared cache tier is only useful if the sites holding it are close in network terms. That turns fiber adjacency into a siting requirement rather than a convenience and it is the strongest available argument for how USDC selects land.
Three diverse paths with a round trip target under ten milliseconds, as described on the Global Network page, make cross-site cache reuse and session migration practical.
Cache movement is a bandwidth consumer, not a rounding error. Sites have to be provisioned for east to west traffic between pods, not only for north to south traffic to the internet.
Land selection favours parcels on dense fiber routes over the lowest cost acreage. A cheaper parcel that adds latency between sites removes the reason the footprint exists.
A customer can be routed to whichever site has capacity while the session context follows them, which converts a multi site footprint from an operational burden into a scheduling advantage.
The site boundary is where the cache tier changes from a performance feature to a capacity feature.
Cross site reuse works for prefix reuse and for moving a session to where capacity exists. It does not work for a tight prefill and decode loop split across two cities.
Within a site, cache moves at fabric speed. Between sites it moves at backbone speed and the workload has to tolerate that difference.
Do not split a prefill–decode loop across two cities.
Bring the prompt reuse profile and the sites you need to reach. USDC will plan the cache tier and the fiber around both.
Let's plan your campus
with confidence.
vLLM and Mooncake published benchmark results on Codex traces • Mooncake Store and LMCache documentation. Benchmark figures are measured by their authors on their hardware and are not USDC results. Inter-site targets are from the USDC Global Network page.
