Taking an LLM Apart, Tensor by Tensor

Every quantization paper I've read tells you the aggregate numbers — "INT4 achieves 99.2% of bf16 perplexity." Useful, but it hides the distribution. Some tensors quantize perfectly; others are a single outlier channel away from falling apart entirely.

Weight Atlas by alesha-pro is the first tool I've seen that asks the right question: which tensors in this model can be compressed losslessly, which will break, and why?

It's an interactive pan-zoom canvas where a whole checkpoint is laid out as every weight tensor, with every number measured, not estimated. The author ran it on Qwen3.8-27B — 27.78 billion parameters across 1,199 tensors — and computed the actual SQNR (signal-to-quantization-noise ratio in dB) for INT8 per-channel, INT4 group-128, and FP8 e4m3 for each one.

Beyond quantization error, it also measures distribution shape (kurtosis, skew, sparsity), outlier fractions beyond 3/4/6 sigma, dynamic range histograms, row/column amax ratios (the outlier-channel problem), and top singular values with stable rank decay. Cross-tensor scatter plots let you see, for instance, whether tensors with heavy tails are the ones that fail INT4 — and yes, they are.

The architecture overview is equally revealing: a "wall" view showing every layer stacked by role (embedding → attention → MLP → head), aligned so the 3:1 attention-to-MLP rhythm and fragile rows read at a glance. The full model took 116 seconds to analyze on one GPU.

This is the kind of tool that should exist for every major release. Before you ship an INT4 quant of a frontier model, you should be able to see which tensors are holding on by a thread. Weight Atlas makes that visible.