Quick Answer: You can run a 284-billion-parameter DeepSeek-V4-Flash model on a 64 GB CPU-only machine with llama-modchip, an Apache 2.0 fork of llama.cpp. The project reports a 56.56 GiB model file and about 9 to 15 tokens per second in decode. Stock llama.cpp, Ollama, and LM Studio can't load these files, so you need the project's own binary.

Most people assume a 284B model needs a GPU rack. That assumption is mostly right for the dense case. But this model is a Mixture-of-Experts design, and only about 13.17B of its parameters are active parameters for any given token. The llama-modchip README claims that combination, plus some aggressive compression, puts the whole thing in RAM on a desktop. Here's what that claim actually involves, what the numbers mean, and where it breaks.

Why a 284B model normally won't fit in 64 GB

The arithmetic is blunt. At 16-bit precision, 284B parameters need roughly 568 GB. Even at 4 bits per weight you're near 142 GB. Neither fits in 64 GB, and the operating system needs its share too.

The project's own baseline shows the problem. The README says the uncompressed DeepSeek-V4-Flash checkpoint at 76.87 GiB crashes with an out-of-memory error on 64 GB Linux. The fix isn't just "quantize harder." Plain quantization tends to hurt quality on the parts of the network that matter most, so llama-modchip targets specific components instead.

The resulting file is 56.56 GiB, which works out to an average under 2 bits per weight. That's a derived figure, not a claim from the README, but it shows how much work the compression is doing.

What llama-modchip actually changes

The README describes several separate techniques. Each one addresses a different part of the memory budget.

  1. SwiGLU-H2 compression targets the feed-forward expert layers. The project says it shaves 20.31 GiB off the checkpoint.
  2. Fractal-64 (Q2_0, 3.75 bpw) MLA transcoding re-encodes the multi-head latent attention weights at about 3.75 bits per weight.
  3. Expert pruning uses what the project calls L2-quadrature scoring to drop low-value routed experts.
  4. Whole-model tiling maps computation onto AVX-512 registers and the L1, L2, and L3 caches for faster prefill.

The project's own explanation is dense, so think of it this way: the compression makes the model small enough to fit, and the tiling makes it fast enough to be usable. Neither alone is enough.

Fair warning: these are the project's own technical claims. The README points to a paper and an architecture document, but I haven't independently reproduced the compression math or the benchmarks. Treat them as a strong starting point, not settled fact.

The numbers, by hardware tier

The README's tier table is the most useful thing in the document, because it answers "what will I actually get?" Here are the reported figures, compared side by side.

Machine tierRAMRecommended modelReported speed
Developer laptop / mini-PC16–32 GBNemotron-3.5-30B-A3B (14.35 GiB)25.0–60.4 tok/s
64 GB desktop / server64 GBDeepSeek-V4-Flash 284B (56.56 GiB)9.1–14.7 tok/s decode, 44.4 tok/s prefill
Hybrid desktop (CPU + 16 GB GPU)32–64 GBNemotron-30B / Qwen3-30B80.80 tok/s
8-channel Threadripper workstation128 GB+Nemotron-30B111.62 tok/s

Two things stand out. First, prefill and decode are different numbers. Prefill (processing your prompt) runs at about 44 tok/s, while decode (generating the reply) runs at 9 to 15 tok/s. If you paste long documents, prefill dominates your wait. If you generate long answers, decode does.

Second, the 284B model decodes roughly four to six times slower than the 30B model on the same class of hardware. That's expected given the active parameter count, but it changes what the 64 GB machine is good for. It's a capable offline reasoning box, not a chat replacement for a hosted model.

The counter-intuitive part: bandwidth beats cores

The common advice for faster CPU inference is to buy more cores. The README suggests that's the wrong lever for decode.

Decoding a token means streaming the active weights from memory, and that work is limited by how fast your memory can deliver bytes. That's why the 8-channel Threadripper reaches 111.62 tok/s on the 30B model, while the 16-core Zen 5 desktop tops out around 60.4 tok/s. The README credits the Threadripper result to register tiling that saturates 8-channel DDR5 bandwidth. Channel count, not just core count, sets the ceiling.

So before you buy hardware, check your memory configuration. A desktop with two DDR5 channels will hit a bandwidth wall long before a workstation with eight does, no matter how many cores you add.

Where it breaks: the caveats that matter

Here's where most readers will get burned, so read this section before you download anything.

The headline context doesn't match the quickstart. The marketing numbers cite 256K context and a peak of 57.61 GiB resident memory. The quickstart command for the 284B model uses -c 32768, which is 32K. Longer context means a larger KV cache, and that memory has to come from the same 64 GB pool. Start at 32K, measure your resident memory, and only then push higher.

Stock tools won't load it. The project uses a custom tensor type, GGML_TYPE_Q2_0 (ID 40). If you open these files in unpatched llama.cpp, Ollama, or LM Studio, you'll see unknown tensor type 40. You must use the project's binary or the ./modchip wrapper. This is the single most common failure mode you'll hit, and it has nothing to do with your hardware.

The benchmarks are self-reported. The leaderboard is community-submitted, and most numbers come from the project's own test runs. Your results will vary with your memory speed, thermal limits, background load, and the prompt length you use.

The OS needs memory too. A model that occupies 56.56 GiB on disk leaves a small margin on 64 GB. Close browsers and other large processes before loading it. If the machine starts swapping, your tok/s will collapse, and the README's "zero NVMe swap" claim stops applying.

The agent use case: warm KV restore

The part I'd pay the most attention to, if you're building agents, is the KV cache restore. The README reports that re-evaluating a 45-second Claude Code system prompt on CPU is the main bottleneck for agent workflows. The project saves the first slot's KV cache to /dev/shm, a RAM-backed filesystem, and restores it in 13.1 to 32.6 ms on the 284B model.

That's a real cause-and-effect improvement: the expensive prompt evaluation happens once, and later turns skip it. The trade-off is that /dev/shm lives in RAM, so the cached state counts against the same 64 GB budget as the model. That interaction is exactly the kind of thing a reader should measure, not assume.

Getting it running

If you want to try it, follow these steps in order. Don't skip the checks.

  1. Clone the repo or download the single-file binary from the v2.4.0 release. The README notes the binary has zero .so dependencies.
  2. Download the 284B model with hf download from Hugging Face, using the repository named in the quickstart.
  3. Make the binary executable with chmod +x.
  4. Start the server with -c 32768 -t 16 -fa on --port 8080, matching your physical core count for -t.
  5. Watch resident memory and tok/s during a realistic prompt, then raise the context window only if you have headroom.

The README also suggests a one-line readiness check piped from curl into python3. Download the script and read it first. Running unreviewed code from the network, even a short script, is a habit worth keeping.

Frequently Asked Questions

What is llama-modchip?

llama-modchip is an Apache 2.0 licensed fork of llama.cpp focused on running large models on CPUs. Its v2.4.0 release supports the 284B DeepSeek-V4-Flash and 30B Mixture-of-Experts models. It uses custom compression and a custom tensor type, so it can't load files meant for stock llama.cpp.

Can you run DeepSeek on CPU with 64GB RAM?

Yes, according to the project's reported benchmarks. The 284B DeepSeek-V4-Flash model is about 56.56 GiB on disk and reportedly decodes at 9 to 15 tokens per second on a 64 GB machine. Expect the context window to be the main constraint, so start around 32K and measure before going higher.

How do I fix "unknown tensor type 40" in llama.cpp?

You can't fix it in stock llama.cpp, because the model uses a custom tensor type (GGML_TYPE_Q2_0, ID 40) that the upstream project doesn't implement. Use the llama-modchip binary or its ./modchip wrapper instead. Loading these GGUF files in Ollama or LM Studio will fail the same way.

Why does a 284B model fit in 64GB at all?

It fits because only about 13.17B parameters are active per token in this Mixture-of-Experts design, and because compression cuts the stored size to 56.56 GiB. The compression is the key part. Without it, the uncompressed model needs about 76.87 GiB and crashes on 64 GB Linux, according to the README.

Here's what to do this week

If you want to know whether this works for you, don't rely on my numbers or the README's. Run a 32K-context test with a prompt that resembles your real workload, record decode and prefill speeds plus peak resident memory, and compare them with the table above. Then decide whether a 284B model earns its place on your machine. Pass this along to anyone trying to run DeepSeek on CPU without a GPU budget, and tell them to measure before they commit.