Apply the RPBH pre-round inside the 8-bit activation kernel (0.16.1)

README.md

@@ -111,7 +111,7 @@ Artifacts saved before a default changed load with the value they were quantized
 
 | Field | Values | Effect | Runtime cost |
 | --- | --- | --- | --- |
-| `rotation_rounds` | `"auto"` (default), `1` (paper), `2` | An independent first RPBH round in front of the paper round. The paper block `h` is the largest power of two dividing `d`: `d = 3360` gets `h = 32`, `d = 3840` and `d = 8960` get `h = 256`, so one round spreads an outlier over few coordinates. `"auto"` adds the round where `h < 512`. | One extra RPBH pass per activation, shared across the projections of one input. CUDA runs it as one Triton kernel: on an RTX 4090 at `d = 3840` it adds 40-50 µs per call at 256 tokens and 70-75 µs at 4096 tokens (the PyTorch path added 0.5 ms and 3.4 ms). MPS runs it as a Metal kernel (+0.06-0.18 ms at 256 tokens, +1.4-1.6 ms at 4096 on an M2 Max). The fused runtime, Krea 2 and the ExecuTorch Vulkan export (blocks up to 1024) support it. |
+| `rotation_rounds` | `"auto"` (default), `1` (paper), `2` | An independent first RPBH round in front of the paper round. The paper block `h` is the largest power of two dividing `d`: `d = 3360` gets `h = 32`, `d = 3840` and `d = 8960` get `h = 256`, so one round spreads an outlier over few coordinates. `"auto"` adds the round where `h < 512`. | One extra RPBH round per activation, shared across the projections of one input. On CUDA the absmax INT8 activation kernel applies it in the same pass for inputs up to 16384 wide: on an RTX 5090 at 4096-9216 tokens the two-round quantizer takes 0.22-0.49 ms at `d = 9536` and 0.05-0.10 ms at `d = 3840`, at most 4% longer than the one-round quantizer of 0.16.0 (0.16.0 ran the round as a separate kernel: 0.57-1.31 ms and 0.12-0.28 ms). At `d = 3360` (`h = 32`) and 9216 tokens it takes 16-19% longer. A norm or modulation prologue keeps its own kernel: with one, the two-round quantizer takes 15-22% longer than the 0.16.0 one-round quantizer at `d = 9536` and `d = 13568`. Lloyd-Max activations get the round from a separate Triton kernel: +0.03-0.06 ms per call at `d = 3840` and +0.07-0.19 ms at `d = 9536` on the same GPU and token counts. MPS runs it as a Metal kernel (+0.06-0.18 ms at 256 tokens, +1.4-1.6 ms at 4096 on an M2 Max). The fused runtime, Krea 2 and the ExecuTorch Vulkan export (blocks up to 1024) support it. |
 | `activation_quantizer` | `"auto"` (default), `"lloyd_max"` (paper), `"absmax_int8"` | `"auto"` resolves to `"absmax_int8"` at `activation_bits=8` and to `"lloyd_max"` below. `"absmax_int8"` is per-token absmax INT8 of the rotated input, the numerics of the fused runtime's 8-bit activations; the artifact stores no activation codebook. On Z-Image W4A8 it matched 8-bit Lloyd-Max under the same rotation (LPIPS 0.331 against 0.330, paired difference -0.007 to +0.007). Codebook version 3 builds 8-bit Lloyd-Max tables in about 0.1 s per width, including `d >= 7680`, where version 2 does not converge. | CUDA runs absmax A8 as an INT8 GEMM against the packed W2/W4 weights, the same GEMM class as W4A4. On an RTX 4090 at 4096 tokens an absmax W4A8 projection takes 3.4-6.6x less time than Lloyd-Max A8 (for example 1.01 ms against 6.71 ms with 12288 inputs and 3072 outputs). |
 | `adaln_scale` | `"absmax"` (paper implementation), `"mse"` | Per INT4 group, the BF16 scale with the lowest squared error from `absmax x [0.6, 1.15] / {7, 7.5, 8}`. | None: offline only, same packed format. Not recommended: it lowers the weight error but raised the AdaLN output error on 59 of 60 CogVideoX-2B projections (median 10x) and 17 of 32 Z-Image projections. |
 | `output_gain_correction` | `False` (default, paper), `True` | Divides the BF16 row norms by `E<u,Q(u)>` of the weight and activation codebooks (`0.9905` per side at 4 bits, `0.8826` at 2). This helps outputs whose weight rows and inputs are aligned and hurts uncorrelated ones. On Z-Image it raised LPIPS against BF16 at W2A4 (+0.016, 95% interval +0.004 to +0.029, worse on 24 of 30 pairs) and did not lower it at W4A4, W4A8 or W3A3, so no setting enables it by default. | None. |
@@ -145,7 +145,7 @@ against BF16 by 0.022 relative to W4A4 on the same images (-0.034 to -0.011).
 CogVideoX-2B (`d = 1920`, `h = 128`, so `"auto"` adds the round) shows no gain from it at W4A6:
 on 12 VBench prompts the default and paper settings differ by less than the pair-to-pair spread
 on all eight custom-input dimensions and on frame LPIPS against BF16 (0.339 against 0.326,
--0.022 to +0.052), while the default took 7% longer (1656 s against 1547 s) and peaked
+-0.022 to +0.052), while the default took 7% longer with 0.16.0 (1656 s against 1547 s) and peaked
 1.2 GiB higher. Use `rotation_rounds=1` for such models when speed matters.
 
 ## Install

docs/paper-methodology-audit.md

@@ -182,7 +182,7 @@ paper fields.
 
 | Field | Paper value | Deviation |
 | --- | --- | --- |
-| `rotation_rounds` | `1` | Default `"auto"` (a second round where the paper block is below 512; on Z-Image-Turbo W4A4 it lowered LPIPS against BF16 from 0.438 to 0.348, better on 27 of 30 prompt/seed pairs); `2`/`"auto"` composes an independent RPBH (seed `rotation_seed + 0x9E3779B9`) in front of the paper rotation. The product is orthogonal, so Equations (4)-(8) hold in the composed basis; the first round is folded into the weights and applied to activations before the paper-round kernels. |
+| `rotation_rounds` | `1` | Default `"auto"` (a second round where the paper block is below 512; on Z-Image-Turbo W4A4 it lowered LPIPS against BF16 from 0.438 to 0.348, better on 27 of 30 prompt/seed pairs); `2`/`"auto"` composes an independent RPBH (seed `rotation_seed + 0x9E3779B9`) in front of the paper rotation. The product is orthogonal, so Equations (4)-(8) hold in the composed basis; the first round is folded into the weights and applied to activations ahead of the paper round (on CUDA, inside the 8-bit absmax activation kernel). |
 | `activation_quantizer` | `"lloyd_max"` | Default `"auto"`: `"absmax_int8"` at 8 bits, `"lloyd_max"` otherwise. `"absmax_int8"` replaces Equation (7)'s 8-bit codebook with per-token absmax INT8 of the rotated input. The artifact then carries no activation codebook. The version 2 generator (tolerance `1e-13`, `6 * 4**8` iterations) does not converge at 8 bits for `d = 7680`, `10240`; version 3 builds those tables, so paper-recipe A8 is available. |
 | `adaln_scale` | not fixed by the paper | `"mse"` replaces the absmax/7 implementation contract with a per-group squared-error scale search. It minimises the weight error, not the AdaLN output error; on 2026-10-29 it raised the output error on 59 of 60 CogVideoX-2B AdaLN projections. |
 | `output_gain_correction` | `False` | Divides the row norms of Equation (6) by the codebooks' retained energy `E<u,Q(u)>`. Stays off by default: on Z-Image-Turbo (30 prompt/seed pairs, 2026-10-29) it raised LPIPS against BF16 at W2A4 by 0.016 (95% interval +0.004 to +0.029) and lowered it at none of W4A4, W4A8, W3A3. |

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "orbitquant"
-version = "0.16.0"
+version = "0.16.1"
 description = "Calibration-free OrbitQuant for transformer linear projections"
 readme = "README.md"
 requires-python = ">=3.11"

src/orbitquant/__init__.py

@@ -24,7 +24,7 @@
 from orbitquant.quantizer import register_hf_quantizers
 from orbitquant.recipes import recipe
 
-__version__ = "0.16.0"
+__version__ = "0.16.1"
 
 register_hf_quantizers()