Quick Answer: To deploy a trillion-parameter model on Cerebras, you split its layers across multiple WSE-3T wafers, each with 44 GB of on-chip SRAM. Each layer stays on its wafer, so only activations move between chips. For Kimi K3's roughly 3 trillion parameters at 4-bit precision, the layout needs 59 wafers: 35 for linear-attention layers and 24 for dense layers.
Deploying trillion-parameter models is a memory problem before it's a compute problem. A single model's weights can run to terabytes, and one wafer holds 44 GB. Sarah Chieng's Cerebras engineering post walks through how the team handles this gap using Kimi K3 as the example. The math is simple enough to check yourself, and it shows exactly where the limits sit.
Why the weights don't fit on one wafer
Start with the raw payload. At 4 bits per parameter, 3 trillion parameters come to 1.5 TB of weights. Spread across Kimi K3's 93 layers, that's about 16 GB per layer.
That number matters because it sits just under the 44 GB on-chip budget of a WSE-3T wafer. A wafer can hold roughly two layers' worth of weights with room to spare, and one dense layer with a large amount of working space left over. That room is where the rest of the design comes from.
Quantization drives this result, so treat it as a real tradeoff. The example assumes 4-bit weights, following the approach in the Hugging Face implementation. Lower precision shrinks the footprint, but it can cost accuracy on some tasks. Run your own evals before you commit to a precision level.
Layer type decides how many layers fit per wafer
Here's the part most capacity estimates get wrong. You'd expect the weights to set the wafer count. They don't, at least not alone. The KV cache does.
Dense (full-attention) layers store keys and values for every token in every active request. That state grows with context length and concurrency. So the layout puts one dense layer per wafer and gives the rest of the SRAM to its cache.
Linear-attention layers keep a small, fixed-size state per request, no matter how long the context gets. That leaves room for two layers per wafer, since 2 × 16 GB = 32 GB fits in 44 GB.
Apply those rules to Kimi K3's layer mix and the count falls out:
| Layer type | Layers in Kimi K3 | Layers per wafer | Wafers needed |
|---|---|---|---|
| Linear-attention | 69 | 2 (about 32 GB) | 35 |
| Dense (full-attention) | 24 | 1 (about 16 GB, leaving about 28 GB for KV cache) | 24 |
| Total | 93 | — | 59 |
Here's the counter-intuitive takeaway. Adding more linear-attention layers doesn't make concurrency worse, but it also doesn't raise it. The dense layers and their KV cache set the ceiling on how many requests you can serve at once. If you're sizing a deployment, start with the dense layers, not the parameter count.
How the wafers connect into a pipeline
Once the layers are placed, the wafers work like an assembly line. Each wafer runs its layers, then passes its output forward. The weights never leave the chip. Only the intermediate results move.
According to the post, those activations cross between wafers via RDMA over Ethernet, at roughly 2 microseconds per hop. That's the reason the design works. Moving weights between chips would dominate latency. Moving activations is a small payload by comparison.
In the example layout:
- Wafer 1 runs layers 1–2 and sends its activations to Wafer 2.
- Wafer 2 runs layers 3–4 and passes results to Wafer 3.
- The last wafer produces the next token, and the loop continues.
This is pipeline parallelism applied to inference. The catch is that it only pays off when the pipeline stays full, which brings up the most common objection.
The idle-wafer misconception
Here's where most explanations stall. If a token has to pass through every wafer in order, doesn't each wafer sit idle while it waits? Isn't the system effectively running at a batch size of one?
No. Running one request through the pipeline at a time would waste most of the hardware. Instead, the pipeline carries many requests at once. While stage 3 processes a token for request A, stage 2 handles request B, and stage 1 handles request C.
Each stage can also process a micro-batch, a small group of requests, rather than one. The number of requests in flight is roughly the number of stages times the micro-batch size. That gives you a real tradeoff. Smaller micro-batches make each request faster. Larger micro-batches push total throughput higher. Pick based on whether your product cares more about per-user latency or aggregate capacity.
How many concurrent requests one configuration can serve
Concurrency is set by the KV cache on the dense-layer wafers. Each dense wafer has about 28 GB left after its layer, and every in-flight request needs its own cache on each dense wafer.
The calculation has two steps:
- Work out how much KV cache one request uses at your target context length. The example uses a 256k-token average request size, which is an assumption, not a measured traffic profile. Real workloads will skew shorter or longer.
- Divide the total KV cache space across the dense wafers by the per-request cache size.
Plug in your model's per-token KV footprint and the calculation gives you a concurrency estimate. The post doesn't publish a per-request figure, so you'll need your model's exact KV dimensions to finish the math. Don't guess those values. Pull them from the model config.
Most people stop here, but concurrency alone doesn't tell you how efficient the system is. Requests in a pipeline don't wait on each other to finish the way they would in a GPU batch. They stream through continuously. That's useful, but it means you need a different yardstick.
Measure efficiency in tokens per second per megawatt
The post argues that the more meaningful metric is total token throughput per unit of power, such as tokens per second per megawatt, measured at the same output generation speed. Comparing systems at different speeds or power levels gives misleading results.
The example targets 1,000 to 2,000+ tokens per second for multi-trillion-parameter models. Whether that speed matters depends on your use case. Interactive coding agents feel the difference. Offline batch jobs mostly care about cost per token.
Scaling works the same way beyond 3T parameters. Calculate the weight and request-state memory, distribute the layers across wafers, and connect them into a pipeline. Each extra wafer adds both compute and on-chip memory, which is how the approach handles models larger than one chip can hold.
Frequently Asked Questions
what is a WSE-3T wafer?
The WSE-3T is a Cerebras wafer-scale chip with 44 GB of on-chip SRAM. Because the whole model can live on-chip, weights stay put and only activations move between wafers. That's what makes the pipeline layout practical for models in the trillion-parameter range.
how many wafers does a trillion parameter model need?
It depends on the model's size and layer mix. Kimi K3, with about 3 trillion parameters at 4-bit precision, needs 59 wafers in the post's example. The count comes from dividing linear-attention layers by two per wafer, then adding one wafer per dense layer for its KV cache.
how do you deploy trillion-parameter models without moving weights?
Split the layers so each one fits in a wafer's on-chip memory, then chain the wafers into a pipeline. Only activations cross between wafers, at about 2 microseconds per hop over RDMA over Ethernet. You need enough on-chip memory per layer, and you need to leave room for KV cache on the dense layers.
why does the KV cache limit how many requests you can serve?
Every in-flight request needs its own KV cache on each dense-layer wafer. Once that space runs out, you can't admit more requests without changing the layout. Linear-attention layers need only a small fixed state per request, so they don't create the same bottleneck.
Start with the math before you buy the hardware
The core lesson is simple. Divide your model's total parameter bytes by its layer count, check how many layers fit in your device's on-chip memory, then separate dense from linear layers to size the KV cache. Try this on a model you already run this week and note where your numbers differ from the vendor's, because that gap is where your real deployment risk sits. If you want to test a large model without buying wafers, the free API at cloud.cerebras.ai is a low-cost place to start.