AI/ML Quantization QuaRot & SpinQuant FP8 Systems PyTorch

Outlier-Free 4-Bit & FP8 LLM Systems: Hadamard Invariance (QuaRot), Stiefel Rotations (SpinQuant) & Blockwise FP8


In LLM Inference & Speculative Decoding and Sparse Mixture-of-Experts (MoE), we saw how modern serving and training pipelines are governed by the Roofline ratio of HBM bandwidth to Tensor Core compute. Dropping weights, activations, and the KV-cache from BF16 down to INT4 or FP8 immediately quadruples or doubles arithmetic intensity – yet naive low-bit quantization collapses above $6.7\text{B}$ parameters due to emergent activation outliers where a handful of hidden channels spike to $20\text{–}100\times$ the median magnitude.

QuaRot (Ashkboos et al., 2024, arXiv:2404.00456; GitHub) and SpinQuant (Liu et al., 2024, arXiv:2405.16406; GitHub) exploit a clean linear-algebraic symmetry of RMSNorm Transformers: multiplying the residual stream by an orthogonal rotation matrix $H \in \mathrm{O}(d)$ ($H H^\top = I$) and counter-rotating every adjacent weight matrix ($W_{\text{in}}' = W_{\text{in}} H$, $W_{\text{out}}' = H^\top W_{\text{out}}$) leaves the full-precision forward pass mathematically identical while smearing channel-aligned outliers uniformly across all $d$ dimensions.


0. Computational Invariance of Orthogonal Residual Rotations

Recall that RMSNorm normalizes a token vector $x \in \mathbb{R}^d$ strictly by its Euclidean $\ell_2$ norm $\|x\|_2 / \sqrt{d}$ before applying a diagonal elementwise scale $\mathrm{diag}(\gamma)$. Because orthogonal transformations preserve $\ell_2$ norms ($\|x H\|_2 \equiv \|x\|_2$), absorbing $\mathrm{diag}(\gamma)$ into the subsequent linear projection $W$ makes the entire block rotation-invariant:

RMSNorm Orthogonal Invariance & Walsh–Hadamard Outlier Dispersion
$$\mathrm{RMSNorm}(x) W^\top = \frac{x}{\|x\|_2 / \sqrt{d}} \mathrm{diag}(\gamma) W^\top = \frac{(x H)}{\|x H\|_2 / \sqrt{d}} \underbrace{\bigl(H^\top \mathrm{diag}(\gamma) W^\top\bigr)}_{\tilde{W}^\top}$$ $$H_{2^k} = \frac{1}{\sqrt{2}} \begin{pmatrix} H_{2^{k-1}} & H_{2^{k-1}} \\ H_{2^{k-1}} & -H_{2^{k-1}} \end{pmatrix}, \qquad H_d H_d^\top = I_d, \qquad |H_{i,j}| = \frac{1}{\sqrt{d}}$$

Because every entry of a normalized Walsh–Hadamard matrix has magnitude $1/\sqrt{d}$, a single channel outlier of magnitude $M$ in $x$ is redistributed into $\pm M/\sqrt{d}$ across all channels – collapsing fourth-moment kurtosis from $\kappa \gt 500$ down to the Gaussian baseline $\kappa \approx 3$!

1. Unrotated Residual Stream X (Kurtosis κ = 582) Single outlier channel dominates INT4/FP8 dynamic range 40× Spike 95% of normal channels crushed into 0 or ±1 quantization bin 2. Hadamard-Rotated X · H (Kurtosis κ = 3.04) Y = (X H)(H^T W^T) ≡ X W^T with uniform sub-Gaussian channels Full 4-bit [-8, 7] grid utilized across every channel
Figure 1: Effect of orthogonal Walsh–Hadamard rotation $X' = X H$ on activation channel distributions. The $40\times$ outlier spike is spread uniformly across all channels with zero change to the full-precision GEMM output.
Two Non-Obvious Footguns When Rotating Transformers: (1) RoPE Does Not Commute with Random Rotations: Because Rotary Position Embedding (RoPE) applies $2 \times 2$ block rotations $R_{\Theta, m}$ inside $Q$ and $K$, you cannot fuse a global residual rotation $H$ into $W_Q$ and $W_K$ across the RoPE boundary – instead, $H$ is absorbed on the input side ($W_Q' = H^\top W_Q$), and a fast online Walsh–Hadamard transform is applied to $Q$ and $K$ after RoPE to quantize the KV-cache to 4 bits. (2) Cayley Stiefel Parameterization in SpinQuant: When optimizing learned rotation matrices $R_1, R_2 \in \mathrm{SO}(d)$ via gradient descent (SpinQuant), projecting onto $\mathrm{O}(d)$ via SVD every step triggers an expensive host-device sync; use the skew-symmetric Cayley map $R = (I - A)(I + A)^{-1}$ ($A = -A^\top$) instead.

1. Runnable PyTorch Reference: Fast Walsh–Hadamard Transform (FWHT) & W4A4 Error Verification

The self-contained PyTorch script below implements the $O(d \log d)$ in-place butterfly Fast Walsh–Hadamard Transform, verifies exact FP64 invariance, and measures the $2.3\times$ MSE reduction under W4A4 symmetric quantization:

💻 Python: Fast Walsh–Hadamard Transform (FWHT) & Outlier-Free W4A4 QuantizationPyTorch
import math
import torch

def fwht_pow2(x: torch.Tensor) -> torch.Tensor:
    """O(d log d) orthogonal Fast Walsh-Hadamard Transform along last dim (Ashkboos et al., arXiv:2404.00456)."""
    # x: [..., D] where D is a power of 2
    d = x.size(-1)
    h = x.clone()
    step = 1
    while step < d:
        h = h.view(*x.shape[:-1], d // (2 * step), 2, step)
        a, b = h[..., 0, :], h[..., 1, :]
        h = torch.stack((a + b, a - b), dim=-2).reshape(x.shape)
        step *= 2
    return h / math.sqrt(d)

def quantize_sym_int4(t: torch.Tensor) -> torch.Tensor:
    """Per-token / per-channel symmetric 4-bit quantization to [-8, 7]."""
    scale = t.abs().amax(dim=-1, keepdim=True) / 7.0 + 1e-12
    return torch.clamp(torch.round(t / scale), -8.0, 7.0) * scale

if __name__ == "__main__":
    torch.manual_seed(42)
    X = torch.randn(32, 64, dtype=torch.float64) * 0.5
    X[:, 7] *= 40.0  # Inject 40x activation outlier spike in channel 7
    W = torch.randn(32, 64, dtype=torch.float64) * 0.2

    Y_exact = X @ W.T
    X_rot, W_rot = fwht_pow2(X), fwht_pow2(W)
    Y_rot_exact = X_rot @ W_rot.T

    mse_naive = (Y_exact - (quantize_sym_int4(X) @ quantize_sym_int4(W).T)).pow(2).mean().item()
    mse_quarot = (Y_exact - (quantize_sym_int4(X_rot) @ quantize_sym_int4(W_rot).T)).pow(2).mean().item()
    kurt_before = (X.pow(4).mean() / X.pow(2).mean().pow(2)).item()
    kurt_after = (X_rot.pow(4).mean() / X_rot.pow(2).mean().pow(2)).item()

    print(f"[PASS] Full-Precision Hadamard Invariance Diff: {(Y_exact - Y_rot_exact).abs().max().item():.2e}")
    print(f"[PASS] Activation Kurtosis (Before -> After)  : {kurt_before:.1f} -> {kurt_after:.2f}")
    print(f"[PASS] W4A4 GEMM MSE (Naive -> QuaRot)        : {mse_naive:.4f} -> {mse_quarot:.4f} ({mse_naive / mse_quarot:.1f}x lower error!)")
# Output:
# [PASS] Full-Precision Hadamard Invariance Diff: 1.07e-14
# [PASS] Activation Kurtosis (Before -> After)  : 152.8 -> 2.73
# [PASS] W4A4 GEMM MSE (Naive -> QuaRot)        : 0.4083 -> 0.1741 (2.3x lower error!)

2. Low-Bit LLM Quantization Schemes: Head-to-Head

Scheme SmoothQuant (W8A8) QuaRot (arXiv:2404.00456, W4A4KV4) SpinQuant (arXiv:2405.16406) + Block FP8
Outlier Mitigation Mechanism Diagonal channel migration $s_j = \max|X_j|^\alpha / \max|W_j|^{1-\alpha}$ Random orthogonal Walsh–Hadamard $H \in \mathrm{O}(d)$ Cayley-learned Stiefel rotations $R_1, R_2 \in \mathrm{SO}(d)$ + $1 \times 128$ tile scaling
4-Bit Activation & KV Accuracy Collapses at 4-bit (outliers migrate to weights) Retains 99% zero-shot accuracy on 70B models Closes W4A4KV4 gap to within 2.9 pts of BF16 on Llama-3
Runtime Overhead Zero (fused into LayerNorm) 2 online FWHT kernels per block ($O(d \log d)$) Zero extra overhead for $R_1$ (fused offline into weights)