A project called APEX (from SigmanticAI) crossed my desk today and it's one of the few repos that made me stop scrolling. It's an open-source inference chip design — real RTL, real Verilog — that runs Qwen2.5-0.5B on an FPGA. Every silicon value is bit-exact against a golden model. Measured throughput: 0.56 tok/s. Not fast, but that's not the point.
Here's what makes this different from the usual "we designed an AI chip" announcement that evaporates into vapor: the repo is a complete, verification-first transparency artifact. Every block — attention, KV-cache compression, softmax, RMSNorm, RoPE, SwiGLU, residual — exists in RTL and is verified against an executable NumPy reference. Not "close enough." Bit-identical. With mutation-tested testbenches and machine-generated evidence under an anti-fabrication rule.
The architectural bet is equally interesting. Most accelerators treat the KV cache as a software problem — quantize it on the CPU/GPU, store it, hope. APEX puts the KV codec inside the datapath. Keys and values are compressed the moment they're produced and decompressed the moment they're consumed, with an importance unit that watches which parts of the context actually matter and spends bits where they count. Reading speed is designed to stay flat as context grows — not O(n²) with sequence length.
This is one tile — no DRAM controller, no PCIe, no NoC — but the architecture is sized for 7B-class models (head_dim=128 exists in RTL, and Qwen2.5-7B tokens have run through the software-verified pipeline). The paper-architecture that wraps the tile into a full chip is specified with per-number provenance.
The scariest part? This was built with Sigmantic AI's autonomous verification agents — Polaris Engine — that take hardware from natural-language spec to verified RTL. The same trend that made software composable (agent-generated, agent-verified) is now coming for hardware design.