The default VLA routed prefill builds its attention mask from
make_prefix_mask(n_pref, &patch_groups) with visual_segments left
empty; only the routed_prefix debug path applies
make_visual_override_segments. Against the reference implementation of
hunyuan_vl_mot, that shape has three deviations for image prompts: patch
rows also attend causally to BOS/USER/VISION_START (blocked in the
reference), patch rows in image-row r see only the r preceding split
tokens rather than the whole block, and split rows stay purely causal
instead of joining the bidirectional segment.
Following up the note in #2 that a separate issue with the evidence
could be useful. Measured effect on the VLM towers (text-generation path
of #2, same graph builder): pointing prompts on a 5-object scene bound
right-of-center targets to the coordinates of objects to their left
(8/16 in-bbox), while generated object descriptions stayed correct.
Applying make_visual_override_segments — already in the codebase —
raised it to 16/16 with no other change (commit dc10403 in #2). Whether action-head outputs are
measurably affected was not tested; the mask-shape identity between the
generate path and the default routed path is what this issue reports.
The default VLA routed prefill builds its attention mask from
make_prefix_mask(n_pref, &patch_groups)withvisual_segmentsleftempty; only the
routed_prefixdebug path appliesmake_visual_override_segments. Against the reference implementation ofhunyuan_vl_mot, that shape has three deviations for image prompts: patch
rows also attend causally to BOS/USER/VISION_START (blocked in the
reference), patch rows in image-row r see only the r preceding split
tokens rather than the whole block, and split rows stay purely causal
instead of joining the bidirectional segment.
Following up the note in #2 that a separate issue with the evidence
could be useful. Measured effect on the VLM towers (text-generation path
of #2, same graph builder): pointing prompts on a 5-object scene bound
right-of-center targets to the coordinates of objects to their left
(8/16 in-bbox), while generated object descriptions stayed correct.
Applying
make_visual_override_segments— already in the codebase —raised it to 16/16 with no other change (commit dc10403 in #2). Whether action-head outputs are
measurably affected was not tested; the mask-shape identity between the
generate path and the default routed path is what this issue reports.