Mia's AI Lab published a single-node launcher that runs DeepSeek V4 Flash 0731 (EXL3/ExLlamaV3) on one NVIDIA DGX Spark — no second node required. The numbers are worth sitting down for: 384k-token context, 44–47 tok/s decode, exact needle recall at 370k tokens, zero preemptions.
The DGX Spark is NVIDIA's desktop AI supercomputer — a single GB10 GPU with 128 GiB unified memory. The official FP4 build needs two Sparks in TP2 config. This build does it on one, using the sparkinfer (formerly b12x) kernel stack with DSpark K5 speculative decoding (K64 draft model) and an NVFP4 compressed KV cache.
Key details from the repo:
- KV pool: 439,622 tokens on a cold boot (1.14× headroom above 384k)
- Stress test: 370k-token prompt with random-word filler (no prefix-cache cheating) — exact passphrase recall at token ~20,
finish_reason: stop, clean - Prefill: ~1,024 tok/s at start, ~350–614 tok/s past 300k accumulated tokens — full 384k prefill takes ~10 minutes
- End-to-end: 370k-token prompt to answer in 594 seconds
- Caveat: weights alone are ~107 GB; ~110+ GiB free disk required. EarlyOOM must be disabled — the server intentionally holds ~94% of memory
The repo ships a complete Docker recipe with two upstream kernel backports applied as read-only bind-mounts. First boot is slow (pulls image, downloads weights, coalesces TP4→TP1, builds K64 draft, captures CUDA graphs). After that, it runs.
Two things make this signal worth your attention.
First, the context ceiling just moved for local hardware. 384k tokens on a desktop machine — not a cluster, not a cloud instance — is the kind of number that changes what people build. Agent loops that need to hold an entire codebase, a full design doc, or hours of conversation history are no longer cloud-exclusive workloads.
Second, the NVFP4 dual-cache prefill path that this build uses was NaN-ing on prompts ≥ 7 tokens before the kernel backports. The fact that two strategically applied patches turned it into a 370k-token needle-recall machine on a desktop form factor is the kind of engineering that doesn't get enough press.
This is what local inference with frontier weights looks like when you stop pretending a Mac Mini is a server and start using the hardware NVIDIA actually built for this.