Qwen3.8 Flash Next on a Single DGX Spark — 99B at Home
What Shipped
MiaAI Lab published a self-contained recipe for serving Qwen3.8-Flash-Next-NVFP4 — a 99 GB vision-language checkpoint — on a single NVIDIA DGX Spark, at tensor parallelism 1. No second node. No exotic plumbing. One ./start.sh and it serves on :8888 in about 10 minutes.
The checkpoint packs 125B total parameters with 6B active (MoE) and handles text, images, and video. The repo ships a .env.sample tuned for the Spark's 121 GiB unified memory budget, with PLE tables offloaded and memory-mapped to stay inside the envelope.
Why It Matters
The DGX Spark is NVIDIA's desktop AI supercomputer — a single box with Grace Blackwell silicon, not a rack. Running a 99B frontier-class model on one of these, with vision input and multi-turn conversation, is the kind of milestone that redefines what "local AI" means. It's not a distilled toy. It's the real Qwen3.8-Flash-Next, at home, with no cloud dependency.
This follows the same trajectory as DeepSeek V4 Flash on DGX Spark from a few weeks ago — but Qwen3.8 Flash Next is a larger, more capable checkpoint with native vision, and the memory management here is even tighter.
The Numbers
The shipped configuration delivers:
- KV cache: 16.46 GiB = ~992,584 tokens (3.79× a 262k request)
- Startup: 10 min 51 s from cold NVMe read to
/health - Decode: ~27 tok/s sustained
- Concurrency: 5 concurrent ~60k prompts completed cleanly, no watchdog events
- Host headroom: 14-16 GiB MemAvailable during load, never below 3.2 GiB MemFree
The repo also documents YaRN-extended 512k context configurations (achieving ~1,500-1,800 tok/s prefill and up to 1.4M token KV pools), though these push the Spark harder and show some FP8 needle-in-a-haystack regression at extreme lengths.