[nightly] Fix garbage output in VLM nightly tests - #1172
Draft
quic-vishali wants to merge 1 commit into
Draft
Conversation
Root causes: - split_model_io absent from disaggregated compile calls - ctx_len from the config smaller than prompt_len for some models - Incomplete preprocessing for several VLM families in test_generate.py Fixes: - Add split_model_io=True to all VLM example compile calls and nightly pipeline config; required for correct disaggregated prefill/decode on multi-SoC configurations - Increase ctx_len from 1024 to 6000 in the nightly config - Refactor test_generate.py preprocessing into per-family helpers: _prepare_internvl_inputs, _prepare_molmo_inputs, _prepare_unified_processor_inputs, _prepare_qwen_vl_inputs, _prepare_legacy_processor_inputs with correct per-model preprocessing steps Signed-off-by: Vishali Senthilkumar <vishsent@qti.qualcomm.com>
Contributor
|
@quic-vishali Can you please fix the linter issues. Did you take the CI run with these changes? Can you please post the results. Changes overall looks good to me |
quic-hemagnih
requested changes
Jul 17, 2026
quic-hemagnih
left a comment
quic-hemagnih
left a comment
Contributor
There was a problem hiding this comment.
an you please fix the linter issues. Did you take the CI run with these changes? Can you please post the results. Changes overall looks good to me
Contributor
Author
Hi @quic-hemagnih, This PR is currently on hold, and a few VLM models are still pending validation. I'm currently working on another task and will share the validation results once I get back to this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix garbage output observed in VLM nightly tests.
Root Causes
split_model_ioabsent from disaggregated compile callsctx_lensmaller thanprompt_lenfor some modelstest_generate.pyChanges
split_model_io=Trueto VLM example compile calls and nightly pipeline configctx_lenfrom 1024 to 6000_prepare_internvl_inputs_prepare_molmo_inputs_prepare_unified_processor_inputs_prepare_qwen_vl_inputs_prepare_legacy_processor_inputs