Repository navigation
Commit 72b2536
authored
Qualcomm AI Engine Direct - [GenAI Pipeline] PR5: Model preparation & quantization strategy implementations (pytorch#21899)
## Summary
This PR implements the __model preparation__ and __quantization__
strategy implementations, replacing the `NotImplementedError` stubs with
real logic. Each strategy delegates to injectable adapter interfaces
(from PR4) for testability.
### What's included
#### Strategy implementations (2 files + 1 `__init__` fix):
- `ExecuTorchModelPreparationStrategy`: 5-step flow
- `load_model` → `load_tokenizer` (via `ModelLoaderAdapter`)
- `generate_calibration_data` (via separately-injectable
`CalibrationDataAdapter`)
- Optional tokenizer export for on-device runtime
- Chat template extraction from tokenizer (with `extra_options`
fallback)
- Validates input config (`model_name`, `soc_model` required)
- `ExecuTorchQuantizationStrategy`: Full PT2E single-graph pipeline via
`QuantizerAdapter`
- export → make_quantizer → prepare_pt2e → calibrate → convert_pt2e
- Supports `quant_dtype`, `quant_recipe`, and per-channel options via
`extra_options`
- Handles any `Iterable` as calibration data (lists, DataLoaders,
generators)
- Validates calibration data is non-empty before export
- Warns (does not fail) when `training_data` is provided (QAT deferred)
- `strategies/model_preparation/__init__.py`: adds missing
`ExecuTorchModelPreparationStrategy` import to `__all__`
#### Unit tests:
- `test_executorch_model_preparation_strategy.py`
- `test_executorch_quantization_strategy.py`
- `test_default_model_preparation_adapter.py`
- `test_default_model_preparation_adapter.py`
### PR Review Checklist
- All new classes follow single responsibility (one class per file) -
Yes.
- All dependencies are injected via constructor with sensible defaults -
Yes.
- All external calls are behind injectable interfaces - Yes (adapter
pattern).
- Unit tests cover every public method - Yes (100% coverage on strategy
impls).
- Docstrings on all public classes and methods - Yes.
- Type annotations on all function signatures - Yes.
- Logging follows the strategy in the LLD - Yes (info on entry/exit,
debug per step).
### Related PRs
- PR 1: Core data model, engine routing & exceptions:
pytorch#20409
- PR 2: Strategy interfaces & stage wrappers:
pytorch#20795
- PR 3: Pipeline orchestrator:
pytorch#21149
- PR 4: Adapter interfaces, default implementations & dataset providers:
pytorch#21751
- PR 5: Model preparation & quantization strategies: this pr.
- PR 6: Compilation & inference strategy implementations: pending.
- PR 7: Integration & E2E tests: pending.
## Test plan
### Run only tests added in this PR:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
-v
```
### Run only this PR's tests with coverage:
```
python -m pytest \
backends/qualcomm/genai_pipeline/tests/strategies/model_preparation/ \
backends/qualcomm/genai_pipeline/tests/strategies/quantization/ \
--cov=backends/qualcomm/genai_pipeline/strategies/model_preparation \
--cov=backends/qualcomm/genai_pipeline/strategies/quantization \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 161 0 32 0 100%
```
### Run all `genai_pipeline` tests:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ -v
```
### Run all `genai_pipeline` tests with coverage:
```
python -m pytest backends/qualcomm/genai_pipeline/tests/ \
--cov=backends/qualcomm/genai_pipeline \
--cov-config=backends/qualcomm/.coveragerc \
--cov-report=term-missing
```
Result:
```
Name Stmts Miss Branch BrPart Cover Missing
----------------------------------------------------------------------------------------------------------------------------------------------------
backends/qualcomm/genai_pipeline/configs/compilation_input_config.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/compilation_output_config.py 8 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_input_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/inference_output_config.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_input_config.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/model_preparation_output_config.py 12 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_input_config.py 13 0 0 0 100%
backends/qualcomm/genai_pipeline/configs/quantization_output_config.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/calibration_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/datasets/default_calibration_data_adapter.py 26 0 6 0 100%
backends/qualcomm/genai_pipeline/datasets/default_training_data_adapter.py 14 0 2 0 100%
backends/qualcomm/genai_pipeline/datasets/training_data_adapter.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/engine_proxy.py 20 0 4 0 100%
backends/qualcomm/genai_pipeline/exceptions.py 20 0 6 0 100%
backends/qualcomm/genai_pipeline/genai_pipeline.py 99 7 12 1 93% 190-201
backends/qualcomm/genai_pipeline/pipeline_context.py 52 0 14 0 100%
backends/qualcomm/genai_pipeline/pipeline_stage.py 5 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/compilation_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/inference_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/stages/model_preparation_stage.py 14 2 0 0 86% 30, 37
backends/qualcomm/genai_pipeline/stages/quantization_stage.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/compiler_adapter.py 11 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/compilation/executorch_compilation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/device_runner_adapter.py 14 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/executorch_inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/inference/inference_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/executorch_model_preparation_strategy.py 68 0 14 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_loader_adapter.py 9 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/model_preparation/model_preparation_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/executorch_quantization_strategy.py 61 0 18 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantization_strategy.py 7 0 0 0 100%
backends/qualcomm/genai_pipeline/strategies/quantization/quantizer_adapter.py 9 0 0 0 100%
----------------------------------------------------------------------------------------------------------------------------------------------------
TOTAL 593 9 76 1 99%
```1 parent 76434b8 commit 72b2536
15 files changed
Lines changed: 1367 additions & 52 deletions
File tree
- backends/qualcomm/genai_pipeline
- configs
- strategies
- inference
- model_preparation
- quantization
- tests
- configs
- strategies
- model_preparation
- quantization
Lines changed: 8 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
30 | 36 | | |
31 | 37 | | |
32 | 38 | | |
| |||
36 | 42 | | |
37 | 43 | | |
38 | 44 | | |
| 45 | + | |
39 | 46 | | |
40 | 47 | | |
41 | 48 | | |
Lines changed: 17 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| |||
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
39 | 47 | | |
40 | | - | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
41 | 51 | | |
42 | 52 | | |
43 | 53 | | |
| |||
48 | 58 | | |
49 | 59 | | |
50 | 60 | | |
| 61 | + | |
51 | 62 | | |
52 | 63 | | |
53 | 64 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
213 | 213 | | |
214 | 214 | | |
215 | 215 | | |
| 216 | + | |
| 217 | + | |
216 | 218 | | |
217 | 219 | | |
218 | 220 | | |
| |||
Lines changed: 6 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
77 | 77 | | |
78 | 78 | | |
79 | 79 | | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
80 | 84 | | |
81 | 85 | | |
82 | 86 | | |
83 | 87 | | |
84 | 88 | | |
85 | 89 | | |
86 | | - | |
| 90 | + | |
| 91 | + | |
87 | 92 | | |
88 | 93 | | |
89 | 94 | | |
| |||
Lines changed: 4 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
4 | 4 | | |
5 | 5 | | |
6 | 6 | | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
7 | 10 | | |
8 | 11 | | |
9 | 12 | | |
| |||
12 | 15 | | |
13 | 16 | | |
14 | 17 | | |
| 18 | + | |
15 | 19 | | |
16 | 20 | | |
17 | 21 | | |
Lines changed: 96 additions & 7 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
| 11 | + | |
12 | 12 | | |
13 | 13 | | |
14 | 14 | | |
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
33 | 44 | | |
34 | 45 | | |
35 | 46 | | |
| |||
94 | 105 | | |
95 | 106 | | |
96 | 107 | | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
97 | 153 | | |
98 | 154 | | |
99 | 155 | | |
| |||
102 | 158 | | |
103 | 159 | | |
104 | 160 | | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
110 | 174 | | |
111 | 175 | | |
112 | 176 | | |
| |||
128 | 192 | | |
129 | 193 | | |
130 | 194 | | |
131 | | - | |
| 195 | + | |
132 | 196 | | |
133 | 197 | | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
0 commit comments