fix: enforce entropy limits on quoted candidates and anchor provenance to the fetched diff

[svozza]

Enforce entropy limits on quoted secret candidates; anchor provenance to the fetched diff

Fixes the three post-schema rejections in
https://github.com/svozza/aceiro-redteam/actions/runs/35323035751 (Aceiro
c7559e2, public rboto fixture). Submission 1 (group_note) was a genuine
model error and stays rejected; a regression pins that.

Root cause

secret_taint.detect_candidates scanned every line with
detect_secrets.core.scan.scan_line (detect-secrets 1.5.0). That ad-hoc API
enables eager search. When the quoted pass finds nothing, the high-entropy
plugins return every unquoted charset run without applying the entropy
limit; the library leaves the limit to the ad-hoc caller so it can print
"why" a string was missed. Aceiro then required the run to be enclosed in ',
" or a Markdown backtick, but never reapplied the limit.

Result on the fixture: eight ordinary public paths became redaction
candidates, e.g. packages/rboto-sns/ (entropy 3.68, limit 4.5) and
crates/rboto-core/ (3.20). Two consequences:

1. Redaction rewrote +++ b/packages/rboto-sns/... headers into
   +++ b/<SECRET_4:...>python/rboto_sns/exceptions.py, and cc_loop.run
   read that redacted diff.patch back for provenance. 18 changed files
   vanished from the verifier's hunk map while changed_files.json and the
   checkout kept the real names, so the correct anchor (exceptions.py:49)
   was rejected as "not inside any diff hunk".
2. Every later mention of a tainted path (crates/rboto-core/src/lib.rs)
   tripped "content reproduces a redacted input candidate" and exhausted the
   submission budget.

A second latent gap fell out of the same mechanism: the eager pass only runs
when the quoted pass is empty, so a low-entropy "..." string on the same
line suppressed detection of a genuine backticked secret.

Change

• secret_taint.py: the two entropy plugins are instantiated directly with
  the existing limits (4.5 base64, 3.0 hex) and no longer go through
  scan_line. Every quote- or backtick-enclosed charset run on a line is
  checked with the plugin's own calculate_shannon_entropy (so the hex
  detector's all-digit penalty still applies) against entropy_limit. The
  pattern and keyword detectors still run through scan_line unchanged.
• cc_loop.py: diff.patch and changed_files.json are read before
  redact_review_inputs, so the verifier's hunk map is derived from the diff
  as fetched. The model-visible copy is still redacted; a test asserts its
  line numbering is identical to the fetched diff, so annotated numbers and
  verifier anchors agree.
• docs/testing.md: one sentence describing the redaction boundary.

No dependency upgrades, no threshold changes, no path or identifier
allowlists.

Security properties preserved

• Secret scanning is still on for pr.json, the diff, and every text file in
  the quarantined head; taints still gate the verifier, transcript, and
  stream capture.
• Keyword and provider-specific detectors (AWS, GitHub, Slack, private key,
  JWT, ...) are untouched and covered by a mixed-quoting test.
• Genuine high-entropy values in backticks, single, and double quotes are
  still detected; unquoted high-entropy runs are still ignored as before.
• Detection is now strictly a subset of before for entropy kinds, except the
  mixed-quoting case, where it is a superset.
• Plaintext values are never logged; tests use synthetic tokens.
• Redaction still applies to +++ headers. A file whose real path contains a
  genuine secret-like token remains unreportable (the finding's path would
  reproduce the taint), which is the fail-closed behaviour that existed
  before.

Regression coverage

tests/test_secret_taint.py::TestEntropyDetectors

• backticked ordinary paths from the incident are not secrets;
• genuine high-entropy secret detected in backticks, ", and ';
• unquoted high-entropy run still ignored;
• low-entropy quoted string cannot mask a backticked secret (was a miss);
• hex detector keeps its own limit and digit heuristic;
• AWS key and keyword detectors unaffected by quoting; mixed line keeps all;
• redact_review_inputs leaves ordinary paths and a real diff header alone.

tests/test_cc_loop.py::TestRunFailureModes::test_provenance_is_anchored_to_the_diff_as_fetched

• a placeholder in a +++ header does not rename the file in the verifier's
  hunk map; the model prompt has no plaintext; annotated line numbers of the
  redacted copy equal the fetched diff.

tests/test_verify_adversarial.py

• group_note remains an unsupported finding field;
• a backticked public path in the input is not a taint and the verifier
  accepts a summary naming it;
• a backticked genuine secret in the input is still a taint and its
  reproduction is rejected.

Existing cases still cover forged paths, path traversal, out-of-hunk lines,
deleted-file lines, and exact secret reproduction.

Every new test was run against the pre-fix sources and failed there
(6 failures in test_secret_taint/test_cc_loop, 1 in the adversarial
corpus).

Validation

uv run --frozen --group test python -m pytest tests/ -p no:cacheprovider -q
2330 passed, 2 skipped

uv run --frozen --group typecheck ty check src/
All checks passed!

Fixture replay (/tmp/rboto-benchmark-full-work/session/context, read-only):
before, 8 candidates and 18 changed files missing from the hunk map after
redaction; after, 0 candidates and 0 missing, exceptions.py:49 valid.

Not included: prompt/model changes and further paid benchmark runs.