Skip to content

Routed prefill mask: vision rows attend causally to pre-vision tokens #3

Description

@vimalnakrani08

The default VLA routed prefill builds its attention mask from
make_prefix_mask(n_pref, &patch_groups) with visual_segments left
empty; only the routed_prefix debug path applies
make_visual_override_segments. Against the reference implementation of
hunyuan_vl_mot, that shape has three deviations for image prompts: patch
rows also attend causally to BOS/USER/VISION_START (blocked in the
reference), patch rows in image-row r see only the r preceding split
tokens rather than the whole block, and split rows stay purely causal
instead of joining the bidirectional segment.

Following up the note in #2 that a separate issue with the evidence
could be useful. Measured effect on the VLM towers (text-generation path
of #2, same graph builder): pointing prompts on a 5-object scene bound
right-of-center targets to the coordinates of objects to their left
(8/16 in-bbox), while generated object descriptions stayed correct.
Applying make_visual_override_segments — already in the codebase —
raised it to 16/16 with no other change (commit dc10403 in #2). Whether action-head outputs are
measurably affected was not tested; the mask-shape identity between the
generate path and the default routed path is what this issue reports.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions