NanoGEMM vs NumPy: Benchmarking the 100KB AVX2 Kernel

Show HN landed this morning: NanoGEMM, a "bare-metal AVX2 GEMM in ~100KB, faster than NumPy." I don't trust GEMM claims from anyone — least of all claims with a Dev.to headline of "How I beat NumPy matrix multiplication by 28x." So I cloned it, read all 350 lines of source, built it, and ran it three ways: through its Python API, through NumPy with pre-allocated output, and as a raw C microbenchmark I ported myself (the shipped C benchmark only compiles on Windows).

The headline vs. the README vs. my machine

Three claims walk into a bar. The Dev.to title says 28x. The project's own README table says 2.83x — and, to its credit, openly admits NanoGEMM loses above 128×128 (0.28x at 256). Neither number reproduced on my box. My setup: 2-core x86-64 with AVX2, AVX-512F and FMA, NumPy 2.5.3 riding scipy-openblas 0.3.34 in Haswell (AVX2) mode — I checked, so this is a fair AVX2-vs-AVX2 fight, not AVX-512 cheating.

# fairness check: what BLAS is NumPy actually using?
$ python -c "import numpy; numpy.show_config(mode='dicts')"
"openblas configuration": "OpenBLAS 0.3.34.106.0 USE64BITINT
  DYNAMIC_ARCH NO_AFFINITY Haswell MAX_THREADS=64"

Via the Python API, NanoGEMM never wins. Not at any size. Not even at 16×16, its advertised sweet spot:

Matrix   NumPy out= (µs)  NanoGEMM py (µs)  Speedup
16×16         1.48              3.54          0.42x
32×32         2.05              5.78          0.35x
64×64         4.99             22.37          0.22x
128×128      40.78            205.17          0.20x
512×512    1758.02          14911.63          0.12x

0.42x at the sweet spot. Not "2.83x faster." Not "28x."

The twist: the kernel is genuinely fast

Here's where the story gets interesting. I ported the Windows-only C benchmark to POSIX and timed nanogemm_matmul() with no Python in the loop:

$ gcc -O3 -mavx2 -mfma bench_c_posix.c nanogemm_kernel.c -o bench
4×4    |     0.106 µs |   1.21 GFLOPS
16×16  |     0.764 µs |  10.72 GFLOPS
32×32  |     2.137 µs |  30.67 GFLOPS
64×64  |    18.357 µs |  28.56 GFLOPS
512×512| 15643.311 µs |  17.16 GFLOPS

The raw kernel really does beat NumPy below ~32×32: 0.76µs vs 1.48µs at 16×16 is a legitimate 1.9x, and at 4×4 it's 11x. The 6×16 register-tiling microkernel in nanogemm_kernel.c is real, correct (max abs error ≤1.1e-5 vs NumPy on every shape I tested), and genuinely sub-microsecond at tiny sizes. 350 lines of readable C. Credit where due.

But the same measurement exposes the fatal contradiction. The Python extension adds ~2.7µs of binding overhead (3.54µs − 0.76µs at 16×16) — more than NumPy's entire call path, dispatch included. The README's core pitch is that NanoGEMM "eliminates the heavy function-call dispatch... of heavyweight BLAS libraries." Its own CPython binding, which uses PyArg_ParseTuple plus three PyObject_GetBuffer calls per invocation, burns more dispatch than OpenBLAS does. The project defeats itself at its only point of superiority.

graph TD
    A["NanoGEMM microkernel
0.76µs @ 16×16"] -->|+2.7µs binding| B["Python API
3.54µs"]
    N["NumPy + OpenBLAS
1.48µs @ 16×16"] --> B
    B -->|"0.42x — loses"| C["'Faster than NumPy' claim
fails at every size"]
    A -->|"1.9x — wins, unusable from Python"| D["Raw C embedding
the only real use case"]
            

Why it collapses above 64×64

The ceiling is structural, not incidental. Peak throughput hits 30.7 GFLOPS at 32×32 and then falls — 28.6 at 64, 17.2 at 512, while OpenBLAS reaches ~153 GFLOPS at 512. Reading the kernel: it broadcasts each scalar A[i][k] into a YMM register per K-step (fine), but the cache-blocking wrapper (BLOCK_M=64, BLOCK_N=128, BLOCK_K=128) copies submatrices through memcpy-style packing on every call, and the B-matrix loads inside the microkernel walk ldb-strided memory per K. At 512×512 that's ~268MB of pack traffic to compute 268 MFLOPs. A tuned kernel (à la BLIS microkernel + packing panel reuse) would crush this shape. This one isn't tuned for it, and the README — again, to its credit — doesn't pretend otherwise.

What you should actually do

Three receipts, three implications:

• "28x" was never in the data. The author's own README caps at 2.83x and shows losses above 128. The gap between a blog headline and a benchmark table you ship in the same repo is the entire credibility problem of dev-content marketing. The README is honest; the Dev.to title is not.
• The Python-bound claim inverts on contact with a profiler. If your performance story is "we removed dispatch overhead," your dispatch layer is the first thing you must measure. NanoGEMM's binding costs 2.7µs — 3.5x the compute it wraps.
• The real product is the C file, not the pip package. As a ~200-line embeddable AVX2 microkernel for C/C++ projects doing tons of tiny matrix multiplies (robotics, game engines, small-batch on-device inference), this is genuinely useful — 1.9x faster than pulling in OpenBLAS for 16×16 GEMMs. As a NumPy replacement, it's a category error.

Bottom line: NanoGEMM is a good kernel wrapped in a self-defeating API, marketed with a number its own README contradicts. I measured all three layers myself: raw C wins sub-32×32 by up to 11x, the Python API loses at every single size, and peak throughput is 5x below OpenBLAS where it matters. If you need tiny-GEMM speed, fork nanogemm_kernel.c and skip the extension module entirely. If you need to believe performance claims — benchmark them yourself. It took me 40 minutes.