pd-bridge Splits DeepSeek-V4-Flash Across CUDA and Metal
What Shipped
A repo called pd-bridge appeared on GitHub yesterday with a result that shouldn't work as cleanly as it does: DeepSeek-V4-Flash — 284B total, 13B active, 256 routed experts — prefilled on two NVIDIA DGX Sparks running vLLM in FP8, then decoded on a single Mac Studio M3 Ultra running MLX in MXFP4. No RDMA, no Thunderbolt, no shared framework. Plain 10 gigabit Ethernet.
The trick: prefill/decode disaggregation normally requires both engines to agree on a KV cache format. These two never can — CUDA paged cache on one side, MLX block layout plus DeepSeek's rotating windows and compressor pools on the other. So pd-bridge doesn't transfer a cache at all. It computes the decoder's finished cache on the prefill machine using the decoder's own projection and pooling math, writes it straight into the Mac's prefix-cache store, and the decoder wakes up thinking it cached the prompt itself. Verified 313/313 bit-exact against a native forward pass. Payload: ~10 KB per token.
The Receipts
From RESULTS.md, all measured 2026-09-06 with a verdict header (X-PD-Bridge) so a silent native fallback can never pose as a bridged number:
- 236K-token cold prompt: 732.3 s native on the Mac alone → 187.6 s bridged = 3.9×
- 79K tokens: 205.8 s → 56.4 s (3.65×) | 17K tokens: 42.6 s → 17.2 s (2.5×)
- Decode rate unchanged: 23–25 tok/s both ways — decode never left the Mac
- Judged quality eval: 5/5 on the bridged leg, same as native
Why It Matters
Every home-inference result I've covered — DeepSeek-V4-Flash on one DGX Spark, Qwen3.8 Flash Next on one Spark — chased the same question: how big a model can one box hold? pd-bridge quietly changes the question. Stop maximizing the box; put prefill where compute is cheap and decode where bandwidth is cheap, and let a homely Ethernet cable join hardware that will never agree on a cache format. The prefill engine is now 65% of wall time — which is exactly where you want the bottleneck.
The honest asterisks, which the author volunteers unprompted: reference implementation, not a library. One model, pinned stacks, monkey-patched private internals of both engines, and a validated envelope of roughly 20K–105K tokens. But the architecture — cache reconstruction instead of cache transfer, across vendors, over 10GbE — is the part that travels.