DeepSeek V4.1 Flash Teardown: 552B Params, 510GB Weights

DeepSeek dropped DeepSeek-V4.1-Flash today — a multimodal MoE claiming a 552B backbone, 196B of "Engram" ngram memory, 8B active params during prefill, and a 1M-token context. Hugging Face comments immediately asked: is this still a "Flash" model? I didn't want to download 510GB to find out, and I didn't have to. You can audit a safetensors release from its headers alone. Here's what 96,085 tensor records told me.

The audit: 48 headers, zero weight downloads

A safetensors file puts its full tensor index in the first few hundred bytes. So I pulled just the headers of all 48 shards with HTTP Range requests — a few megabytes of traffic for the entire model's table of contents:

# Range-request each shard, parse header length, then the JSON index
req = urllib.request.Request(url, headers={"Range": "bytes=0-16777215"})
data = r.read(16777216)
n = struct.unpack("<Q", data[:8])[0]
hdr = json.loads(data[8:8+n].decode())

Measured, not claimed:

• Total download: 510,296,708,312 bytes = 510.3 GB across 48 shards, 96,085 tensors.
• MoE expert weights: 297.5B params on disk (FP4-packed — see below; 544B logical).
• Engram memory: 203.07B params on disk, stored in BF16 (~406 GB) — that's 80% of the entire download.
• Dense backbone + attention: 7.18B params. Vision encoder: 0.41B.

For calibration: the previous V4-Flash was 284B and fit in ~160 GB. This one needs 510 GB. On disk the breakdown is stark:

graph LR
    A[510.3 GB download] --> B[Engram lookup tables
406 GB · BF16 · 80%]
    A --> C[384 routed experts x 40 layers
316 GB · FP4]
    A --> D[dense + attention
14 GB · FP8/BF16]
    A --> E[vision encoder
0.8 GB]

The Engram 196B claim checks out exactly

The report says Engram is "196B parameters, sparsely accessed via token-based lookup." The config makes this testable: two Engram layers (IDs 1 and 14), each with engram_num_embeddings: ~384,000,000 entries over a 16,000,000-token ngram vocabulary, up to 4-grams, with engram_head_dim: 256.

The math: 2 tables × 384M entries × 256 dims ≈ 196.6B params. My header-derived count for the Engram tensors: 203.07B (the extra ~6.5B is padding entries — 384,006,168 and 384,016,682 aren't round numbers for nothing). Verdict: the claim is real, down to the last decimal. This isn't a cache or a side file — DeepSeek shipped a ~200B-parameter compressed ngram index as first-class model weights, BF16, on disk, looked up sparsely at inference.

Config forensics: the CED architecture is visible in plain text

You don't need the tech report PDF to confirm the Causal Encoder-Decoder design — it's in config.json:

"kv_source_layer_ids": [2, 8, 14, 20],
"index_source_layer_ids": [2, 8, 14, 20, 24, 28, 32, 36],
"compress_ratios": [0,0, 2,2,...,2 (19x), 1,1,...,1 (18x), 0,0],
"sliding_window": 128

Read the compress_ratios array: 43 entries — 2× compression for the 20 encoder layers, 1× for 18 decoder layers, full fidelity for the last 2. And kv_source_layer_ids shows all four global-KV projections come from encoder layers ≤ 20 — exactly the CED structure the README describes. Global KV is projected from four encoder snapshots instead of being carried per decoder layer, which is how they get to the claimed 890 bytes per token of persistent KV cache (with FP4 main KV — E2M1 format per the quantization config). I can't run this model to verify the 890 B/token number, but every structural decision it depends on is right there in the config.

Other confirmations: YaRN factor 16 stretching 65K native context to 1M; 384 routed experts + 1 shared per MoE layer across all 40 layers (I counted the expert tensors: 94464/6 = 15,744 = 384 × 41); 35.4M params per expert; three MTP layers for DSpark speculative decoding.

"Flash" is now a pricing tier, not a size class

One thing I won't prettify: calling a 552B-backbone model with 510GB of weights "Flash" is a category rename. The HN thread made the same point within hours — the original V4-Flash fit on dual Spark / Strix Halo boxes; this one requires FP4 community quants and multi-machine setups. What "Flash" means in 2026 is cheap per token: 8B active during prefill, 16B during decode. The density story is real. The small-model story is dead.

And still — this is the right trade. 510GB on disk but 8B active per token means input-heavy agentic workloads (the thing everyone runs now) get frontier-adjacent behavior at commodity serving cost, and the whole thing ships under an MIT license. The Receipts: benchmarks in the README put it at 90.6 on Terminal-Bench 2.1 and 74.2 resolved on DeepSWE v1.1 — above Opus-5.0's 89.1/74.0 on the same harness, per DeepSeek's own numbers, so salt accordingly. Notably, DeepSeek's own pricing docs still had no v4.1 entry when I checked this morning — release velocity is outrunning their own documentation.

Bottom line

Header-level auditing is the cheapest serious research tool in the open-weights world: ~10MB of HTTP Range requests bought me a 748B-parameter claim set, two exact confirmations (Engram 196B, 384×35.4M experts), and one honest correction to the narrative (Flash no longer means small). The open-weights position keeps winning on a specific axis — you can do this to an open release and you can't do it to anything closed. V4.1-Flash is not a small model pretending to be cheap. It's a big model that found a way to be cheap anyway. That's a better trick.