fix(contract): surface run warnings in the SARIF projection - #285
Open
rohanpoudel2 wants to merge 7 commits into
Open
fix(contract): surface run warnings in the SARIF projection#285rohanpoudel2 wants to merge 7 commits into
rohanpoudel2 wants to merge 7 commits into
Conversation
A scan whose target drifted mid-run records a warning, but the SARIF projection only ever built toolExecutionNotifications from deferred coverage rows, and only when completeness was not complete. A drifted target leaves completeness at complete, so no invocations block was emitted and the warning had no route into SARIF at all. Run warnings now reach the projection independently of completeness, deduplicated against the deferred reasons that already notify verbatim.
|
Note To use Codex here, create a Codex account and connect to github. |
1 similar comment
|
Note To use Codex here, create a Codex account and connect to github. |
# Conflicts: # sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py # sdk/typescript/_bundled_plugin/scripts/workbench_db.py
# Conflicts: # sdk/typescript/_bundled_plugin/scripts/finalize_scan_contract.py
Re-finalizing an already-complete scan and backfilling legacy finding details both rewrite exports/results.sarif through finalize_scan without the warnings the scan recorded, so a regenerated projection dropped the notifications the first completion had written. Both paths now read the persisted warnings back, matching what regenerated exports already do. Reconcile the projection with the coverage-derived execution status added in openai#601. Success still comes from coverage completeness and the invocation is still always emitted, while run warnings join the notification list whatever the completeness. Invocation validation accepts an unsuccessful execution and an absent notification list, since a complete scan with no warnings notifies nothing.
rohanpoudel2
force-pushed
the
fix/sarif-run-warnings
branch
2 times, most recently
from
August 22, 2026 00:36
999cffa to
3778da4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #251.
A scan whose target drifted mid-run records a warning, but that warning never reached the SARIF
projection, so a SARIF consumer could not tell the results described a stale tree. Drift leaves
coverage.completenessatcomplete— the scan really did review everything it set out to review,the tree simply moved underneath it — and notifications were only ever built from
coverage.deferredand only when completeness was not
complete. Run-level warnings therefore had no route into SARIFregardless of completeness.
This projects durable run warnings into
runs[].invocations[].toolExecutionNotifications, includingtarget-drift warnings on otherwise complete scans.
Changes
coverage completeness. Deferred coverage already contributes its reason verbatim, so the same text
is not notified twice.
exports/results.sarifthroughfinalize_scanwithout the warningsthe scan had recorded, so a regenerated projection silently dropped the notifications the first
completion wrote: re-finalizing an already-complete scan, and backfilling legacy finding details.
Both now read the persisted warnings back, matching what regenerated exports already did.
coverage or run warnings.
Reconciliation with #601
#601 landed in this area after this branch was opened and rewrote the same block. Both contracts are
preserved:
emitted, including the synthesized reason for incomplete coverage with no deferred entries.
toolExecutionNotificationsis attached only when there is at least one notification, so acomplete, warning-free scan keeps the bare successful invocation fix: Set SARIF success from coverage #601 produces.
coverage, and an absent notification list, since a complete scan with no warnings notifies nothing.
No public CLI surface changes: no new commands, flags, accepted values, environment variables, or
defaults.
Testing
From
sdk/typescript/, withmainmerged in atf222faf:bun test --timeout 30000 tests-ts/scan-recovery.test.tsandtests-ts/cli-export.test.ts— 68passed, 0 failed, 547 assertions across the two files.
pnpm run test— 1,577 passed, 29 skipped, 0 failed, 21,641 assertions across 1,606 tests in87 files.
pnpm run types— passed.pnpm run format— passed, all matched files use Prettier style.pnpm pack— passed.git diff --check— passed.Coverage added for combined incomplete coverage and run warnings without duplicate notifications, for
warnings surviving a repeated finalization of an already-complete scan, and for warnings surviving
legacy finding-detail backfill. The last two delete the generated SARIF before re-triggering the path,
so they fail without the corresponding fix. The existing complete-with-drift and regenerated-export
cases are retained, and the clean-complete assertion now requires the notification key to be absent.
Tests use synthetic fixtures and temporary directories. No live scan, network, or credentials were
required. Post-push GitHub Actions checks remain pending.
Two local check failures are unrelated to this branch and reproduce identically on unmodified
mainin the same environment, which was verified against a clean checkout:
audit:prodreports onepre-existing high advisory, and
check:packagefails in its installed-package smoke because thepacked tarball resolves
@openai/codexat a prerelease version the public registry does not serve.package.jsonandpnpm-lock.yamlare byte-identical tomainon this branch, so no dependencychanged here.
Risk and rollout
SARIF gains warning notifications only when durable run warnings exist, so warning-free projections
stay byte-compatible. Findings, seals, fingerprints, and coverage completeness are unchanged, and the
execution-success semantics introduced by #601 are unchanged. Consumers that already read
toolExecutionNotificationsfor deferred coverage will now also see run warnings there, which is theintent. The two regeneration paths become consistent with initial completion rather than emitting a
projection that silently omits recorded warnings. No migration or configuration change is required.
Public disclosure review