Repository navigation
ci: add advisory ImpactGate reports #1138
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
75cdac4
0a84761
79e9324
523fed7
587d06e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| 'stash': patch | ||
| --- | ||
|
|
||
| The supply-chain skill names `pip` among the Dependabot ecosystems the repository monitors, and says that a pip requirements file installed in CI should pin every package with a hash and be installed with `pip install --require-hashes --no-deps`. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| # Appended to ImpactGate defaults. Production generators remain eligible. | ||
| ignore: | ||
| - '**/dist/**' | ||
| - 'dist/**' | ||
| - '**/target/**' | ||
| - 'target/**' | ||
| - '**/generated/**' | ||
| - 'generated/**' | ||
| - '**/*.generated.*' | ||
| - '**/*.gen.*' | ||
| - '**/*.d.ts' | ||
| - '**/node_modules/**' | ||
| - '**/vendor/**' | ||
| # Committed Go bindings carry DO NOT EDIT generated headers. | ||
| - '**/*_stash.go' | ||
| - '**/*_gen.go' |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| # Explicit policy prevents a checkout-local root config changing CI analysis. | ||
| cognitive_max: null |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,103 @@ | ||
| # Advisory ImpactGate toolchain: every package pinned, every distribution hashed. | ||
| # CI installs it with `pip install --require-hashes --no-deps`, so nothing here | ||
| # resolves at install time and a transitive dependency cannot float. | ||
| # | ||
| # The top-level pins are impact-gate==0.4.1 and lizard==1.23.0 (Lizard 1.24.0 is | ||
| # excluded by ImpactGate's own metadata). To regenerate after changing them, from | ||
| # the repository root: | ||
| # | ||
| # printf 'impact-gate==0.4.1\nlizard==1.23.0\n' \ | ||
| # | uv pip compile - --generate-hashes --universal --python-version 3.12 \ | ||
| # --no-header -o .github/impact-gate/requirements.txt | ||
| # | ||
| # then restore this header. Any change here changes the baseline cache key. | ||
| impact-gate==0.4.1 \ | ||
| --hash=sha256:9d9c8fb4c6f2fa56559d0530ff82569bc6d42b28ff5a77103e504e2470f198c6 \ | ||
| --hash=sha256:e6dab6abec925151bd4dbffff92d4a40f742d561a6f8527b3fbe15dd68b25518 | ||
| lizard==1.23.0 \ | ||
| --hash=sha256:e9111e35c8a5f2e00d55cab318fca3504622991417411ea16cf46874fb752f42 \ | ||
| --hash=sha256:ed75cd45f086a2f51d6be64b0149b71bda820f92f95e30898254528bb949f795 | ||
| # via impact-gate | ||
| pathspec==1.1.1 \ | ||
| --hash=sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a \ | ||
| --hash=sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189 | ||
| # via lizard | ||
| pygments==2.21.0 \ | ||
| --hash=sha256:2363c69b61c4a97c838da3b130dcd6468f4848992b21a82f2a63ec34377137d9 \ | ||
| --hash=sha256:610ca751c9bc2492b38eb9a38a7fbc93edbbb2d7182edaf34e66ae493dee5c8c | ||
| # via lizard | ||
| pyyaml==6.0.3 \ | ||
| --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ | ||
| --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ | ||
| --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ | ||
| --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ | ||
| --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ | ||
| --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ | ||
| --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ | ||
| --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ | ||
| --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ | ||
| --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ | ||
| --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ | ||
| --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ | ||
| --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ | ||
| --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ | ||
| --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ | ||
| --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ | ||
| --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ | ||
| --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ | ||
| --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ | ||
| --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ | ||
| --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ | ||
| --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ | ||
| --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ | ||
| --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ | ||
| --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ | ||
| --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ | ||
| --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ | ||
| --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ | ||
| --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ | ||
| --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ | ||
| --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ | ||
| --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ | ||
| --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ | ||
| --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ | ||
| --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ | ||
| --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ | ||
| --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ | ||
| --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ | ||
| --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ | ||
| --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ | ||
| --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ | ||
| --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ | ||
| --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ | ||
| --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ | ||
| --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ | ||
| --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ | ||
| --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ | ||
| --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ | ||
| --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ | ||
| --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ | ||
| --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ | ||
| --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ | ||
| --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ | ||
| --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ | ||
| --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ | ||
| --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ | ||
| --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ | ||
| --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ | ||
| --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ | ||
| --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ | ||
| --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ | ||
| --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ | ||
| --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ | ||
| --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ | ||
| --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ | ||
| --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ | ||
| --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ | ||
| --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ | ||
| --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ | ||
| --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ | ||
| --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ | ||
| --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ | ||
| --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 | ||
| # via impact-gate |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # Appended to ImpactGate defaults. Production generators remain eligible. | ||
| ignore: | ||
| - '**/dist/**' | ||
| - 'dist/**' | ||
| - '**/target/**' | ||
| - 'target/**' | ||
| - '**/generated/**' | ||
| - 'generated/**' | ||
| - '**/*.generated.*' | ||
| - '**/*.gen.*' | ||
| - '**/*.d.ts' | ||
| - '**/node_modules/**' | ||
| - '**/vendor/**' | ||
| # Test FILE exclusions do not remove inline Rust test modules. | ||
| - 'test/**' | ||
| - '**/test/**' | ||
| - 'tests/**' | ||
| - '**/tests/**' | ||
| - '**/__tests__/**' | ||
| - '__tests__/**' | ||
| - '**/__mocks__/**' | ||
| # Other fixture-only trees are under tests/__tests__/integration-tests above. | ||
| # eql-domains/src/fixtures is the production catalog DSL: keep it eligible. | ||
| - '**/cli/scripts/fixtures/**' | ||
| - 'fixtures/**' | ||
| - '**/test-fixtures/**' | ||
| - '**/integration-tests/**' | ||
| - 'e2e/**' | ||
| - '**/e2e/**' | ||
| - '**/*.test.*' | ||
| - '**/*.spec.*' | ||
| - '**/*_test.go' | ||
| - '*_test.go' | ||
| - '**/eql-tests-macros/**' | ||
| # Shared test harness and injectable fake backend, including historical homes. | ||
| - '**/test-kit/**' | ||
| - '**/internal/guest/testing.go' | ||
| - '**/*_stash.go' | ||
| - '**/*_gen.go' | ||
| - '**/__fixtures__/**' | ||
| - '**/test_*.py' | ||
| - '**/tests.rs' | ||
| - '**/proptest_invariants.rs' | ||
| - '**/internal/testpolicy/**' | ||
| - '**/internal/testusers/**' |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| name: Change impact | ||
|
|
||
| on: | ||
| pull_request: | ||
| push: | ||
| branches: [main] | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| # A newer push to the same pull request supersedes the older run. Main pushes | ||
| # are grouped per SHA instead: one shared main group cancelled a merge's run | ||
| # before it saved that SHA's baselines, leaving PRs on it to rebuild. | ||
| concurrency: | ||
| group: impact-${{ github.event_name == 'push' && github.sha || github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| impact: | ||
| name: Change impact (advisory) | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 20 | ||
| defaults: | ||
| run: | ||
| shell: bash | ||
| env: | ||
| BASE_REF: refs/remotes/origin/${{ github.base_ref || github.ref_name }} | ||
| steps: | ||
| - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6 | ||
| with: | ||
| fetch-depth: 0 | ||
| persist-credentials: false | ||
|
|
||
| - uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6 | ||
| with: | ||
| python-version: '3.12' | ||
|
|
||
| - name: Install analysis tools | ||
| run: python -m pip install --require-hashes --no-deps -r .github/impact-gate/requirements.txt | ||
|
|
||
| - name: Check reporting behavior with the real CLI | ||
| run: python -m unittest discover -s scripts/tests -p test_impact_gate.py -v | ||
|
|
||
| - name: Pin and identify target branch | ||
| id: target | ||
| env: | ||
| TARGET_BRANCH: ${{ github.base_ref || github.ref_name }} | ||
| BASE_SHA: ${{ github.event.pull_request.base.sha || github.sha }} | ||
| run: | | ||
| # Keep one logical target ref across push/PR events, pinned to the | ||
| # event's commit even if the remote branch advances while queued. | ||
| git update-ref "$BASE_REF" "$BASE_SHA" | ||
| echo "BASELINE_DIR=$RUNNER_TEMP/impact-baseline" >> "$GITHUB_ENV" | ||
| python - <<'PY' | ||
| import hashlib | ||
| import os | ||
| with open(os.environ['GITHUB_OUTPUT'], 'a') as output: | ||
| key = hashlib.sha256(os.environ['TARGET_BRANCH'].encode()).hexdigest() | ||
| output.write(f'key={key}\n') | ||
| PY | ||
|
|
||
| # The fallback stays within one tool/policy AND target branch. Never use a | ||
| # repository-wide fallback: another release branch has different history. | ||
| - name: Restore baselines | ||
| id: restore | ||
| uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4 | ||
| with: | ||
| path: ${{ env.BASELINE_DIR }} | ||
| key: impact-${{ hashFiles('.github/impact-gate/**', 'scripts/impact-gate.py', '.github/workflows/impact.yml') }}-${{ steps.target.outputs.key }}-${{ github.event.pull_request.base.sha || github.sha }} | ||
| restore-keys: impact-${{ hashFiles('.github/impact-gate/**', 'scripts/impact-gate.py', '.github/workflows/impact.yml') }}-${{ steps.target.outputs.key }}- | ||
|
|
||
| - name: Validate or build baselines | ||
| id: prepare | ||
| env: | ||
| REFRESH: ${{ github.event_name == 'push' && steps.restore.outputs.cache-hit != 'true' }} | ||
| run: | | ||
| args=() | ||
| if [[ "$REFRESH" == 'true' ]]; then args+=(--refresh); fi | ||
| python scripts/impact-gate.py prepare --base "$BASE_REF" --cache-dir "$BASELINE_DIR" "${args[@]}" | ||
|
|
||
| # Any rebuilt pair is saved, including on pull requests: a PR's cache is | ||
| # scoped to its own ref, so it cannot replace main's, and later pushes to | ||
| # the PR reuse it instead of rebuilding both scopes again. | ||
| - name: Save baselines | ||
| if: steps.prepare.outputs.rebuilt == 'true' | ||
| uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4 | ||
| with: | ||
| path: ${{ env.BASELINE_DIR }} | ||
| key: ${{ steps.restore.outputs.cache-primary-key }} | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This save can fail permanently for a given SHA. If an exact-key cache exists but fails validation (policy or hash mismatch, corrupt files), Generated by Claude Code |
||
|
|
||
| - name: Report pull request impact | ||
| if: github.event_name == 'pull_request' | ||
| run: python scripts/impact-gate.py report --base "$BASE_REF" --cache-dir "$BASELINE_DIR" | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cancel-in-progress: trueapplies to main pushes too. If two merges land close together, the first run is cancelled before "Save main baselines" and no cache is written for that SHA. PRs based on it then fall back to an older ancestor cache throughrestore-keys, becausevalid_pairaccepts any ancestor, and they are scored against a stale baseline. Considercancel-in-progress: ${{ github.event_name == 'pull_request' }}.Generated by Claude Code