Repository navigation
fix(artifacts): reject escaped credentials without changing completion policy - #1301
Merged
Merged
Conversation
…ld a rejected report (#1217) The artifact secret gate matched configured credentials byte for byte, so a value holding a quote, a backslash, a tab or a line break never matched the JSON that escapes it, and a verified report could publish it. The gate now also scans a JSON publication's decoded strings, joined by NUL so adjacent fields never form one phrase. Configured values are also found percent-encoded, with Markdown backslash escapes, or broken across lines. The verifier, synchronization and the rejected-report check now match every credential an agent can see: the configured api_key_env variables and the allowlisted variables that hold a credential, not only names like *_API_KEY. The value of ULTRAFUZZ_PROVIDER_CREDENTIAL_ENV_NAMES, a list of names, is no longer treated as a credential. The Modal public bundle, eval diagnostics and recovery lifecycle checks reuse the same scan. A final report the gate rejects is withheld instead of failing the run: the report verifier tolerates failure in the engine, the failure records reason_code ARTIFACT_SECRET_GATE_REJECTED, the run ends succeeded when everything else did, status reports report-withheld-secret, and `resume --retry-failed` regenerates the report. Any other report verification failure, and a withheld report under --require-complete, still fail the run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ped escaped credentials (#1217) Review follow-up: - The unchecked report fallback rescanned a rejected draft against the current environment, so once the credential it matched was unset or rotated, `ultrafuzz report` could publish the withheld draft. A report attempt whose failure records ARTIFACT_SECRET_GATE_REJECTED now stays unavailable until a fresh attempt replaces it. - The line-break check compared a configured value only against the Markdown-unescaped text, which had already lost the value's own backslash. The text as written is now checked too. - The compileSmithersWorkflow continueOnFail count expects the report verifier's own continue policy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Configured credentials containing JSON escape characters could pass the publication gate and appear in reports. This extracts the security fixes from #1291 while preserving the existing campaign completion policy.
The gate scans decoded JSON values with field boundaries intact, includes configured credential names from the run, and keeps a secret-rejected report unavailable after its credential is rotated or unset. A failed final-report verifier still fails the campaign; the proposed success-with-withheld-report behavior is separated into its own PR.
Validation: artifact tests (336 passing), publication/unverified-report/final-report support tests (87 passing), focused generated-verifier tests (5 passing), runtime build and test compilation, strict ESLint, and formatting. Focused synchronization regressions cover rejection, credential rotation, and retry recovery.
Closes #1217. Replaces the security portion of #1291; original implementation commits retain their authorship.
The independent completion-policy proposal is isolated in draft #1302.
The PR appears safe to merge, with no actionable issues found.Summary
This PR extends publication checks to decoded JSON strings and common escaped forms of configured credentials. It also records final-report secret rejections so unchecked reads cannot expose the rejected draft after a credential changes.
No actionable issues were found. The review checked source and regression tests; tests were not run.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart TD A[Agent output] --> B[Check bytes and decoded JSON strings] B --> C{Secret found?} C -->|No| D[Publish verified output] C -->|Final report rejected| E[Save rejection reason] E --> F[Keep unchecked draft unavailable] F --> G[Operator retries report task] G --> AReviews (1) · Last reviewed commit: "fix(runtime): preserve completion policy..." · Reviewed by Greptile