Skip to content

test-dynamo(0711): add dynamo causal-lm test infra + fix gemma3 and qwen3_moe subfunction export - #1174

Draft
vbaddi wants to merge 30 commits into
quic-enable-dynamo-for-causallmfrom
dynamo_causal_lm_test_infra
Draft

test-dynamo(0711): add dynamo causal-lm test infra + fix gemma3 and qwen3_moe subfunction export#1174
vbaddi wants to merge 30 commits into
quic-enable-dynamo-for-causallmfrom
dynamo_causal_lm_test_infra

Conversation

@vbaddi

@vbaddi vbaddi commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Adds tests/dynamo/causal_lm/ — a test suite for the use_dynamo=True export path (PR #1169). Covers 32 architectures across 16 feature columns: export, FP16/BF16/MXFP6/MXINT8 compile, CPU parity (HF PT == QEff PT == ORT), HW parity (ORT == QAIC FP16), CB, blocking-KV, sampler, prefix caching, and multi-device (CCL). HTML + JSON report generated on every run.

Parallelism model:

  • Export/compile/CPU-parity lanes are CPU-bound and run fully parallel under pytest-xdist -n 4. Each test writes to a blake2b-keyed workdir (arch + feature + precision hash) so workers never collide and cross-run compile cache reuse works automatically.
  • Runtime lanes (QAIC generate, CB, blocking, sampler, prefix caching) carry @pytest.mark.xdist_group(name="qaic-runtime"), which xdist places on a single worker — serialising them. Each test additionally acquires a device from a flock-based DevicePool before touching hardware, so external concurrency (e.g. two pytest sessions running simultaneously) is also safe. Multi-device tests atomically acquire num_devices and release all holds on failure to prevent deadlock.

Two model fixes required for the subfunction lane to pass:

  • gemma3: switch RMSNorm to select_interface and bridge cache via QEffSlidingWindowCache.from_legacy_cache.
  • qwen3_moe: clone split expert params to remove aliasing in QEffQwen3MoeExperts.__qeff_init__.

vbaddi and others added 29 commits July 10, 2026 11:50
Squash of prior dynamo work (16 WIP commits) into a single commit
to simplify rebase onto mainline. Original history preserved at
tag backup/before-rebase-20260501-212703.

Contents:
- Enable torch.dynamo-based export with inline compiler across causal
  LMs, VLMs, embedding, ASR models.
- Subfunction extraction / nested compile region utilities.
- Torch patches and custom-op utils consolidated under QEfficient/utils.
- Dynamo-specific tests and e2e example.

(cherry picked from commit 70f0cac)
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
…r models

Signed-off-by: vbaddi <vbaddi@qti.qualcomm.com>
(cherry picked from commit 087f8d4)
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: vbaddi <vbaddi@qti.qualcomm.com>
(cherry picked from commit 5fa0d57)
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
…ck previous commit, if this fails

Signed-off-by: vbaddi <vbaddi@qti.qualcomm.com>
(cherry picked from commit 2ca7b45)
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
…mmit (nit(0502), if this fails

Signed-off-by: vbaddi <vbaddi@qti.qualcomm.com>
(cherry picked from commit 22d6385)
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
(cherry picked from commit 575545a)
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
(cherry picked from commit 806f2cf)
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
(cherry picked from commit 6dd6362)
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
(cherry picked from commit 3266f7c)
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>
…en3_moe subfunction export

Signed-off-by: vbaddi <vbaddi@qti.qualcomm.com>
@vbaddi vbaddi self-assigned this Jul 11, 2026
@vbaddi vbaddi added wip Work in progress onnx.dynamo 1.23 Release 1.23 Features labels Jul 11, 2026
Signed-off-by: vbaddi <vbaddi@qti.qualcomm.com>
@vbaddi vbaddi changed the title test-dynamo(0711): add dynamo CausalLM test infra + fix gemma3 and qwen3_moe subfunction export test-dynamo(0711): add dynamo causal-lm test infra + fix gemma3 and qwen3_moe subfunction export Jul 12, 2026
@vbaddi
vbaddi changed the base branch from main to quic-enable-dynamo-for-causallm July 13, 2026 06:03
amarquic added a commit to amarquic/efficient-transformers that referenced this pull request Jul 14, 2026
Creates tests/dynamo/ following the same structure as PR quic#1174:
  - conftest.py: markers, dynamo_workdir fixture, xdist-safe result capture,
    HTML/JSON report generation
  - utils/hash_manager.py: blake2b-keyed workdirs (collision-safe under -n auto)
  - utils/hardware_manager.py: file-lock QAIC device pool for HW lanes
  - utils/report_generator.py: HTML + JSON coverage matrix output

Extends DynamoModelSpec in model_registry.py with two new flags:
  weight_free_supported     - True for all validated causal LM architectures
  weight_free_cb_supported  - True for all architectures where CB is also verified
  Both flags False for VLM text-side models (out of scope for this PR)

Adds tests/dynamo/causal_lm/test_weight_free.py with 4 CPU-only lanes:
  1. test_weight_free_export        - meta-device export produces ONNX + weight_spec.json
  2. test_weight_free_no_embedded_weights - ONNX has no large embedded initializers
  3. test_weight_free_spec_coverage - all spec keys resolve in prepared_checkpoint/
  4. test_weight_free_cb_export     - continuous_batching=True export produces artifacts

All lanes are @pytest.mark.dynamo_export (CPU, no QAIC hardware) and
@pytest.mark.regular. Run with:
  pytest tests/dynamo/causal_lm/test_weight_free.py -m 'dynamo_export' -n auto

Signed-off-by: Amar <amarshar@qti.qualcomm.com>
@quic-amitraj
quic-amitraj force-pushed the quic-enable-dynamo-for-causallm branch 5 times, most recently from 7e7959d to 1f893d5 Compare July 23, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

1.23 Release 1.23 Features onnx.dynamo wip Work in progress

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants