dots3-note W6a: the DENSE vision tower, on a SERVED image request

[localai-org-maint-bot]

dots3-note's DENSE vision tower, on a SERVED image request.

This is the first brick on MODEL-MM-dots3-note-dots3-note-for-causal-lm whose
output a client can ask for. W3 wrote host maths, W4a/W4b put the two attention
geometries on the decode path and W5 put the MoE layers there; all of them ended
at a ctest binary, because until recently there was no engine path by which a
vision tower could be fed from a production entry point at all. #2398 put the
multimodal input pipeline in the runner and #2481 made the server's chat seam
dispatch on the ARCHITECTURE, so the remaining work was this model's own half.

An image_url chat request now travels the whole chain:
ApiServer::handle_chat_completions -> the architecture-dispatched multimodal
chat seam -> GPUModelRunner::execute_mm_encoder -> ModelRegistry::EmbedMm ->
ModelRegistry::Forward -> Dots3NoteModel::ForwardDevice, which reads
mm->inputs_embeds in place of the embedding lookup. kDots3NoteFactory grew
encode_mm and embed_mm, and supports_multimodal went FALSE -> TRUE with
both halves behind it. mrope_prompt_positions stays null, because upstream's
Dots3NoteForCausalLM is SupportsMultiModal, SupportsPP and not
SupportsMRoPE.

Registering this architecture is a NEW translation unit plus one
REGISTER_VLLM_MM_CHAT line, with zero edits to any shared table. dots3-note is
the second architecture to reach the registry #2481 built, which is what that row
was for.

The arm

patch_embed -> blocks 0-24 -> post_trunk_norm -> the patch_merger adapter
folding 4x1536 to 5120. Each block is a fused attn.qkv with no bias, a per-head
q_norm/k_norm applied BEFORE a 2-D vision RoPE, bidirectional attention, and
the three-tensor SwiGLU routed through layers::MlpGateUpMethodBase by merging
fc1 and fc3 into one [2I, E] operand at load. Beside it is a dots3 image
processor with its own resized_size, its own PER-CHANNEL normalization and its
own pre_pixel_shuffle patch order, none of which the Qwen3-VL processor can
stand in for.

The tower shares no code with qwen3_vl_vision and the reason is in the header:
the two agree on the block outline and on almost nothing below it. What IS shared
is every seam underneath — the vt:: ops, dense_attn's device glue,
dense_loaders' weight readers and MlpGateUpMethodBase.

The released checkpoint still refuses, and that is correct

17 of its 42 vision blocks are pyramid MoE and 1960 of its 2195
vision_encoder.* tensors are W6b's, so the tower refuses at block 25 naming the
brick, the tensors stay a named deferral, and the deferral table's brick moved
W6 -> W6b. Video is refused to W7, audio to W8, the blockwise-FP8 vision arm
to W9. This is W3's polarity applied to the second tower, not a new exception.

One behaviour changed under measurement rather than by design. Refusing only
inside encode_mm throws in the engine's busy loop: it stopped AsyncLLM, and
every later request, TEXT ONES INCLUDED, came back 500. The served-request gate
caught it. The chat FACTORY now asks the same refusal, so the install returns a
REFUSING seam and an image request is HTTP 400 naming the block while text keeps
serving. The gate asserts both halves.

The gate is a CONSISTENCY gate

Spec §6.4 option B stands: the checkpoint is 298.67 GB fp8 against 119-122 GiB
hosts, vLLM cannot be run on it on any hardware this project owns, and no
denominator exists. Correctness is argued against an independent in-test
double-precision reference written from nvidia/vision.py and
nvidia/vision_attention.py, sharing no helper with the implementation. It
establishes that two implementations agree. It does NOT establish that either
matches vLLM, and no performance number is claimed on any axis.

Measured: relative 8.44e-3 against a 2.0e-2 bound, with a qk-norm/rope order swap
reaching 5.9e-2.

Every upstream anchor names the SHA it was read at. dots3_note is BEYOND our
parity pin 5559679229; the sources were read at 9035151d6, and upstream has
already moved — vision_attention.py is 477 lines there and 494 at vLLM main
7a100bb61.

Reachability is held by mutation

Deleting the tower call, deleting the .mm read, or deleting the loader's
materialization call site each reds the served gate. Replacing the tower with a
correctly-SHAPED constant reds EXACTLY ONE case — the two-different-images
logprob one — while status, prompt_tokens and completion_tokens all still
pass, which is the measurement behind having that case at all. Deleting the
production call site leaves the tower's own unit gate green at 8/8 and 4981
assertions, which is why both files exist. Binary hashes and the byte-for-byte
restores are recorded in spec §4.11.6.

Records

Spec §7 assigned "the FP32-scale FP8 formula" to W6 and §8's R5 was filed against
W6. The row's own W2 census contradicts both: the released bf16 checkpoint
carries 37944 BF16 + 62 F32 tensors and no scale tensor at all, and the -fp8
sibling that does ship them is already refused by name as W9. Both moved to W9;
the risk is unchanged and is not waived, only its owner moved.
model-matrix.md:124 still read "W2 LANDED" and omitted W3, W4a, W4b-1/2/3 and
W5, so it is brought current in the same change.

#2512's scope prose says the dense arm ends in "pixel shuffle -> adapter", while
the released vision_config sets adapter_type: "patch_merger" — upstream's own
name for the arm that SKIPS that permutation, because pre_pixel_shuffle: true
moved the 2x2 regrouping into the preprocessor and the RoPE. The issue's own
tensor inventory, adapter.{ln_q, mlp.0, mlp.2}, is PatchMergerAdapter's state
dict and agrees with the committed fixture, so the inventory is right and the
word is loose. Recorded in spec §4.11.1 rather than reconciled silently.

Fresh review repairs (this PR, on top of the reviewed head)

The headline fix was incomplete, and the cascade it removed was still
reachable. Moving the MoE refusal into the chat factory fixed ONE condition.
The install-time predicate was still a strict SUBSET of the VT_CHECKs
EncodeMmDots3NoteForCausalLM makes on a served request: an ALL-DENSE
checkpoint whose adapter_out_dim is not the text hidden_size, or whose
adapter_merge_size is not spatial_merge_size, installed cleanly, served
text, and then threw inside the engine's busy loop on its first image — after
which AsyncLLM::errored_ is set for the life of the process
(async_llm.cpp:584-601) and every later request, text ones included, is dead.
Upstream keeps those two merge keys independent with independent defaults, so
this is a config a checkpoint can carry rather than a shape the parse rules out.
Dots3NoteVisionRefusal now makes the same two comparisons, from a
text_hidden_size copied off the LANGUAGE config at parse — the value the route
actually compares against, which the dead vision_config.hidden_size field it
replaces was not. Two new served cases gate it, each asserting HTTP 400 on the
image AND HTTP 200 on a text request afterwards on the same server; the second
assertion is the one the pre-repair tree cannot pass.

A refusal that could never fire is gone. adapter_merge_size**2 * adapter_in_dim != merged_dim() compared merged_dim() against its own
definition — x != x. It was also the only row in the refusal table naming no
brick, because it stood for no condition. The adapter_merge_size check above
is the real cross-key check that belongs in its place.

A record the code claimed did not exist. The image processor's resize
refusal said, in its comment and in its runtime message, that it was "Recorded
under ## Owed in .agents/specs/dots3-note.md". That section named it
nowhere and no issue tracked it. It is a capability gap and not a corner:
factor is 28 on the released geometry and upstream ALWAYS resizes, so once W6b
lifts the MoE refusal almost no real image is servable. Now recorded under
## Owed naming W8, tracked by #2537, and cited by both the comment and the
message.

What the consistency gate CANNOT see is now part of its evidence. A uniform
+1% scale error on the tower output reads relative 0.0152 against the 0.02 bound
and PASSES — the detection floor for a systematic multiplicative error is ~1.5%.
Replacing exact-erf GELU with the tanh approximation is BYTE-IDENTICAL to the
baseline, because the bf16 store of fc1 absorbs it, so a named formula choice
is below the gate's resolution. Both are in §4.11.6 beside the 8.44e-3 / 0.02
numbers, which stand. §4.11.6 also stops overstating its mutation proof: a
changed binary sha proves the build was not stale, not that the mutation reached
the code under test — M4 moved the tower binary's sha by relinking while its
behaviour was unchanged. The case counts are the evidence.

Two smaller ones. Dots3NoteVisionParams::pre_pixel_shuffle defaulted to true
against upstream's DotsMoEVitConfig default of False; production always goes
through the parser, which was already correct, but the field selects between two
incompatible token orders and a default-constructed params must not claim the
wrong one. And TinyCheckpoint::value_of returned a static empty vector for an
unknown tensor name, which ref::Tower would then index — an instrument that
fails toward garbage instead of reporting itself. It throws naming the tensor.

§4.11.6 (Evidence) also moved after §4.11.5 (Reachability), which it preceded.

The full gate was rerun on the repaired tree: ninja all 1381 targets
NINJA_RC=0, ctest -j 2 702/702 with 0 failed and 7 skipped for absent
checkpoints, CTEST_RC=0. The four dots3 suites are 9/68, 8/4996, 51/6888 and
26/110835, all 0 failed. scripts/agent-preflight.sh rc 0.

#2537 is FILED, not fixed here: the bicubic resize it asks for is W8's, and the
spec's ## Owed names it.

Closes #2512.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: CLAUDE:claude-opus-5 [claude-code]