From 569ca6072ed4b28c5a312ecfafd083091a80cecd Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Thu, 3 Sep 2026 16:49:08 -0500 Subject: [PATCH] sec(crypto-gate): the discovery gate could not read half its corpus (BACKLOG #1172) The ASVS 11.1.3 crypto-inventory gate is `import ast` over `*.py`, so a weak-PRNG draw in the TypeScript extension was invisible to it by construction. It did not merely omit `ide/` from WALK_ROOTS; adding the directory there would have been a no-op that looked like a fix, because the Python AST scanner finds zero `.py` files under it and reports clean. Add a second arm to the same required context. `check_non_python_randomness` walks `ide/` for `.ts`/`.js` randomness sources and diffs them against `NON_PYTHON_INVENTORY` the same bidirectional way, so a weak draw in the shipped extension now blocks a merge. The extension's own negative test could not do that from a leg `ci-gate` does not require. Three properties, each pinned by a test and each proven by a mutation control run against the shipped code rather than asserted: * it FINDS a planted `Math.random()` in a `.ts` file. Deleting the pattern reds 6 tests. * it does NOT flag the legitimate `randomBytes` draw in `ide/src/cspNonce.ts`, which is the hard case: that file names `Math.random` twice in prose to explain why it is unusable. Disabling the comment skip reds the false-positive guard. * it REFUSES an empty walk. A scan reaching no files, and an inventory anchor that has gone missing, both red. Disabling the empty-walk refusal reds 1 test. A weak source has no inventory row available to it by construction. Registering one is the move #1172 names as a disqualified pass for ASVS 11.5.1, so the weak check runs before the inventory diff and no row can silence it. The randomness half is the whole claim the arm supports. The extension's TLS floor is still outside both arms, and PowerShell under `scripts/` is still outside the Python one; that gap stays filed under #1164. Also on #1172, which stays OPEN because its closing act is a vaulted scorecard re-score that no engine checkout may perform: * correct the dead clause at the top of the row. It asserted that the extension mints its CSP nonces from `Math.random()`, which was true when filed and was superseded by PR #422 on 2026-08-17. Struck rather than deleted, because the evidence that the row was once wrong is what stops the next reader trusting the rest of it unchecked. * record the five zero-margin values the item's caveat said its census flags. An AST census over the five walk roots finds 21 CSPRNG call sites outside tests and resolves exactly five distinct values to 128 bits: four in the engine (sandbox request id, upload file id, STOW-RS boundary, WS-Security nonce) and one in the console (CSP nonce). AST rather than grep, which returns 36 lines because a docstring naming a draw matches as readily as a draw. * write the handoff for the two acts that remain: the ASVS 11.5.1 scorecard re-score and the scope-completeness adversarial pass that was never run, naming the engine-side evidence and where it lives without reproducing any vaulted content. Co-Authored-By: Claude Opus 5 --- docs/BACKLOG.md | 41 +++- scripts/security/crypto_inventory_check.py | 234 +++++++++++++++++++-- tests/test_crypto_inventory_scanner.py | 220 +++++++++++++++++++ 3 files changed, 480 insertions(+), 15 deletions(-) diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 4b71b3902..9e3bb6cb7 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -397,7 +397,7 @@ Ordered by value descending, then difficulty ascending (cheapest first at equal | 46 | **#1300** | required-contexts.txt describes the CI gate roll-up as six legs when it gates eight, so the file that exists to answer 'does this block a merge' answers it wrong for the two most-run legs | 6 | 2 | _quick win_ | P1 | Gap stands: ci.yml:2596-2616 names eight needs while .github/required-contexts.txt still says six, and the two omitted legs are the ones that actually report -- webconsole has no if: gate (ci.yml:841-843) and tooling's PR filter includes docs/ (ci.yml:1379). Value 6 survives attack because the workaround is available but not reachable from where a reader stands: the correct answer is written out in ci.yml:2603-2616, yet a session asking 'does a red repo harness tests block me' reads required-contexts.txt, finds no such context, and stops. Difficulty 2 because the remainder is a prose edit plus one additive test on a seam that already exists in both directions -- tests/test_merge_gate_controls.py:186 already reads ci-gate's needs, and test_required_contexts.py:136 already reconciles one checked-in claim against another. | | 47 | **#1040** | Hook deny text is attacker-influenceable output that an agent is instructed to act on, and nothing treats it as such | 6 | 3 | _quick win_ | P2 | The two proven-exploitable surfaces are closed and one helper per class exists in PowerShell, so the residual is applying the same treatment to the Python hooks, of which claim_check.py:134-140 is a confirmed untreated prose interpolation of a newline-capable value. Value drops from 8 because the highest-influence values (a refname into a command, a Write file_path into prose) are now folded, and difficulty drops from 5 because the audit is done and the Python hooks need one small helper rather than the PowerShell one. | | 48 | **#1086** | Rule 3c reads a PowerShell here-string as a command, so a commit message that quotes a disarm key is refused | 6 | 3 | _quick win_ | P2 | No message-flag blanking has shipped: Get-ScannableSegments (scripts/hooks/worktree_gate.ps1:403) blanks only ordinary quoted spans, and its own residual list at :470 names BACKLOG #1086 as the pending change that would remove the accidental multi-line denial. The coupled #1097 recursion gap HAS landed (the flag family is now derived by rule, with the -Com, -Comm, --Com, unicode-dash and /Q/C measurements recorded at :411-540), so the remainder is the banked closed-flag-set patch re-derived against the shipped recursion plus the two undisclosed cases; value 6 because rewording the message is a real if record-corrupting workaround the item itself demonstrates. | -| 49 | **#1172** | research an honest pass for ASVS 11.5.1 -- a 128-bit recovery code a person can still transcribe | 6 | 3 | _quick win_ | P2 | The concrete IDE defect is closed by cspNonce.ts and its negative test, so what remains is the recovery-code shortfall (totp.py:58-60, 31^15 or about 74.3 bits, ten issued by default at settings.py:1798) plus the cross-language inventory question the crypto gate still cannot answer (crypto_inventory_check.py:456-472 excludes ide/ by invariant rather than inventorying it). An operator can only work around the shortfall by setting the count to 0, which is the awkward-workaround band. | +| 49 | **#1172** | research an honest pass for ASVS 11.5.1 -- a 128-bit recovery code a person can still transcribe | 6 | 3 | _quick win_ | P2 | The 2026-08-20 reasoning behind these scores is now largely spent, and the scores are LEFT AS THEY WERE because re-scoring is an assessor decision, not a side effect of an edit. What it said: the concrete IDE defect is closed by cspNonce.ts and its negative test, so what remains is the recovery-code shortfall plus the cross-language inventory question the crypto gate cannot answer. Both remainders have since shipped -- the shortfall by PR #603 (totp.py `_RECOVERY_GROUPS` 3 -> 6) and the inventory arm on 2026-09-03 (`check_non_python_randomness` now walks ide/ instead of excluding it). What is left is the part no engine checkout can do: the ASVS 11.5.1 scorecard re-score in the vault, and the scope-completeness adversarial pass that was never run. Read the item, not this cell. | | 50 | **#1205** | half the ASVS record is prose nothing checks: ~2,000 `file:line` citations, none verified | 6 | 3 | _quick win_ | P2 | The tool ships with the properties the item claims, including the empty-scan refusal, but no baseline exists and nothing invokes it, so the mode cannot go red. Value 6 because wiring it freezes roughly a thousand grandfathered claims so the part of the record a reviewer actually reads cannot decay further, with no product or PHI effect; difficulty 3 because the remainder is not one step -- the vault runs mirrored copies of engine tools, so wiring means mirroring this one, generating the baseline with --print-keys, adding the leg, and bringing the new copy under the drift gate that already exists for exactly this failure. | | 51 | **#1229** | the worktree gate blanks double-quoted spans FIRST, so a stray quote inside single-quoted words straddles and deletes the live command between them | 6 | 3 | _quick win_ | P2 | Attacked the shipped claim by driving the real hook, then by mutation. The ORDERING limb is genuinely shipped: the two ordered regexes are gone, replaced by Remove-QuotedSpans (worktree_gate.ps1:347-388), and it is CALLED on the scan path at :654 -- the only call site -- not merely present. Landed in c7f0e308 naming #1229. Behaviour measured with a governed repos file and a positive control (plain `git -C checkout main` -> DENY): straddle DENY, mirrored DENY, quoted commit message ALLOW, unterminated quote DENY. Non-vacuity proven by mutation: a scratchpad copy with the two regexes restored at the call site flips ONLY the straddle arm to ALLOW, so the shipped scanner is what closes it. The TEST limbs are shipped too -- tests/test_worktree_gate_quote_straddle.py asserts the straddle DENIES and pins the mirrored, unterminated and false-positive controls, and it is registered in tests/tooling_manifest.txt:109. What is NOT shipped is the fix criterion the item itself wrote down: the scanner respects quote-vs-quote but ignores backslash escaping, so the same straddle class is still reachable. Two shapes ALLOW at HEAD and both really run the middle command in bash (verified with an inert marker); removing the backslashes makes both DENY, and a single-sided escape is harmless, which mirrors the item's own boundary finding one level down. Scoring the remainder only: value 6, a real gap with no clean workaround in the same shipped guardrail, but per section 0 and the item's own scope this is a local maintainer-workstation guardrail whose synopsis declines to be a security boundary -- no product, engine or PHI effect, and nothing deployed. Difficulty 3: one PowerShell function plus test arms, no mypy or store backends, but shell escape semantics differ inside single quotes, double quotes and unquoted text and the fail-closed unterminated case must survive. Verdict is partly_shipped rather than confirmed_shipped because the scope call is the only judgment here and a wrongly-closed item is invisible forever; if a reviewer rules the escape hole a separate filing, the remainder text above is the filing. | | 52 | **#1278** | run Routers and Handlers in the subprocess sandbox by default | 6 | 3 | _quick win_ | P2 | The default is unchanged at config/settings.py:1294, so on the shipped default Router and Handler code runs in the engine's own address space alongside the store and every connection's in-flight data, while the isolation mode itself is built and exercised. Value 6 because the gap is real but an informed operator can already set [sandbox].mode=subprocess, an awkward rather than absent workaround (it costs a restart and fail-closed refusal of live enrichment); difficulty 3 because the change is one default plus a release note, but the samples and tests/test_sandbox.py must actually be RUN under the new default rather than inspected, and the flip changes behaviour on every config-dir serve path. | @@ -9845,7 +9845,25 @@ Nothing here touches the TLS/FTPS context in the same module, which was already > item's own caveat says was never run; recording the five zero-margin values its census already > flags; and the cross-language randomness inventory arm. > -> **Filed 2026-08-08 - not started. RESEARCH item: the goal is an HONEST pass, and "cannot honestly reach pass" is a valid finding.** ASVS **11.5.1** (L2) currently scores **partial**. The pinned verb sets a 128-bit floor on every non-guessable random value. TOTP recovery codes are CSPRNG-drawn but reach only about 74.3 bits, and the IDE extension mints its webview CSP nonces from `Math.random()`. +> **PARTIAL 2026-09-03 -- THE INVENTORY ARM SHIPPED AND THE ZERO-MARGIN VALUES ARE NAMED. THE ROW +> STAYS OPEN on the same ground as before: the closing act is a scorecard re-score, which lives in +> the vault and belongs to a Tracker seat.** Three of the four remainders above are now done and one +> is untouched. **Built:** `check_non_python_randomness` in +> `scripts/security/crypto_inventory_check.py` walks `ide/` for `.ts`/`.js` randomness sources and +> diffs them against `NON_PYTHON_INVENTORY` bidirectionally, riding the SAME required context +> (`crypto-inventory (ASVS 11.1.3 discovery gate)`, `.github/required-contexts.txt:118`) as the +> Python arm -- so a weak draw in the extension now blocks a merge, which the shipped extension test +> could not do from a leg `ci-gate` does not require. A weak source has NO inventory row available to +> it by construction, which is this item's disqualified move refused in code rather than in prose. +> An empty walk and a missing inventory anchor both red, so the arm cannot report a clean result on a +> corpus it never read. Three mutation controls were run against the shipped code, not asserted: +> deleting the `Math.random` pattern reds 6 tests, disabling the empty-walk refusal reds 1, and +> disabling the comment skip reds the false-positive guard. **Recorded:** the five zero-margin values +> below. **Corrected:** the dead IDE clause at the top of this row, which had gone on asserting a +> defect the row's own body records as retired. **NOT done and named as such:** the scorecard +> re-score and the scope-completeness adversarial pass -- see *The handoff* below. +> +> **Filed 2026-08-08 - not started. RESEARCH item: the goal is an HONEST pass, and "cannot honestly reach pass" is a valid finding.** ASVS **11.5.1** (L2) currently scores **partial**. The pinned verb sets a 128-bit floor on every non-guessable random value. ~~TOTP recovery codes are CSPRNG-drawn but reach only about 74.3 bits, and the IDE extension mints its webview CSP nonces from `Math.random()`.~~ **[BOTH CLAUSES SUPERSEDED -- struck, not deleted, because the evidence that this row was once wrong is what stops the next reader trusting the rest of it unchecked. The recovery-code clause was superseded by PR #603 on 2026-08-25 (`_RECOVERY_GROUPS` 3 -> 6, 148.63 bits raw and 142.98 multiplicity-adjusted). The IDE clause was TRUE when filed and was superseded nine days later by PR #422 (`c49a8a7b`, 2026-08-17), which added `ide/src/cspNonce.ts` -- 18 bytes of `randomBytes` from `node:crypto` -- and consolidated the twelve hand-copied generators into it. This item's own research then measured it false on 2026-08-20 at `fc54d169`, and the top of the row went on asserting it for another two weeks. RE-MEASURED 2026-09-03 at `fd44b0f17`: zero `Math.random(` call sites anywhere under `ide/` in `.ts` or `.js`, against a positive control in the same run of 6 bare `Math.random` occurrences and 110 files walked, so the walk reached the corpus. None of the 6 is a call: five sit in comments (two in `cspNonce.ts`, which names the generator to explain why it is unusable) and one is a test NAME string. The top of this row had gone on asserting the defect in the present tense while its own body recorded the retirement; the two now agree.]** > Verdict: research > Closing-act: scorecard-rescore @@ -9854,7 +9872,7 @@ Nothing here touches the TLS/FTPS context in the same module, which was already **The pinned verb.** "Verify that all random numbers and strings which are intended to be non-guessable must be generated using a cryptographically secure pseudo-random number generator (CSPRNG) and have at least 128 bits of entropy. Note that UUIDs do not respect this condition." -**What holds it short today.** The CSPRNG half holds everywhere -- session tokens, TOTP secret, WebAuthn challenge, OIDC state and PKCE verifier, console CSP nonce, all from stdlib `secrets`. The floor is what fails. Recovery codes are three groups of five characters over a 31-character alphabet, 31^15, with `[auth].mfa_recovery_code_count = 10` at `config/settings.py:1726`. The IDE hit is twelve files with one `Math.random()` call each. Nothing gates either: no entropy floor check and no start-time refusal, and the crypto-inventory gate is `import ast` Python-only, so it structurally cannot see the second (`scripts/security/crypto_inventory_check.py:43-46`). Two checks the verb's own wording forces came back clean and add nothing: every `uuid4` here is a database identifier rather than a bearer secret, and non-CSPRNG `random` appears only in `generators/` and behind the keyed HMAC in `anon/`. +**What holds it short today.** The CSPRNG half holds everywhere -- session tokens, TOTP secret, WebAuthn challenge, OIDC state and PKCE verifier, console CSP nonce, all from stdlib `secrets`. The floor is what fails. ~~Recovery codes are three groups of five characters over a 31-character alphabet, 31^15, with `[auth].mfa_recovery_code_count = 10` at `config/settings.py:1726`. The IDE hit is twelve files with one `Math.random()` call each. Nothing gates either: no entropy floor check and no start-time refusal, and the crypto-inventory gate is `import ast` Python-only, so it structurally cannot see the second (`scripts/security/crypto_inventory_check.py:43-46`).~~ **[ALL THREE SENTENCES SUPERSEDED -- struck, not deleted. Recovery codes are SIX groups of five since PR #603 (`auth/totp.py:67`); the twelve `Math.random()` files were consolidated into `ide/src/cspNonce.ts` by PR #422 on 2026-08-17, so that sentence describes no shipped state; and the gate is no longer Python-only, so it no longer "structurally cannot see the second". Two gates now exist where the paragraph says none: `tests/test_recovery_code_entropy.py` derives the entropy floor from the shipped constants, and `check_non_python_randomness` in `scripts/security/crypto_inventory_check.py` reads `.ts`/`.js` under `ide/`. What is still ungated is a START-TIME refusal -- nothing in `serve` declines to boot on a sub-floor configuration -- and that remains true.]** Two checks the verb's own wording forces came back clean and add nothing: every `uuid4` here is a database identifier rather than a bearer secret, and non-CSPRNG `random` appears only in `generators/` and behind the keyed HMAC in `anon/`. **The research question.** What code shape honestly clears 128 bits while staying transcribable by a person reading a printed sheet under time pressure -- and is there one that gets there without making the recovery path unusable, which is the property the current shape protects? A second question, smaller: is a CSP nonce a "non-guessable" value in this verb's sense at all, and if it is, what would a cross-language crypto inventory have to look like to see a non-Python draw? @@ -9872,6 +9890,23 @@ Nothing here touches the TLS/FTPS context in the same module, which was already **Still not an honest pass**, and the trap has moved. Arguing the code is adequate because lockout and rate limiting bound guessing remains disqualified -- all that bounding is real and was verified rather than dismissed (`auth/policy.py:102-103`, `auth/service.py:2180-2190`, `api/auth_routes.py:362-363`), and all of it is irrelevant, because the verb states a floor on the VALUE. It is the more dangerous trap precisely because its premises survive checking. The move now occupying the vacated position is sharper: setting `[auth].mfa_recovery_code_count = 0`, a shipped, documented, validator-blessed knob whose zero value `docs/CONFIGURATION.md:588` describes as disabling recovery codes. It stops the sub-floor value being minted so a scan finds nothing under the floor, with an aggravating factor the retired dodge never had -- it does not hide the credential, it DELETES the lost-authenticator recovery path. And taking the bare-minimum widening buys a pass that survives only if one reading wins, with the record not saying which reading it depended on. **One caveat on this research itself, stated because it is load-bearing:** this requirement received only ONE adversarial review, on the cheap-flip axis. The scope-completeness axis was never run, its census is a `secrets`-and-`os.urandom` pattern that can only find GOOD draws, and its own honest reporting already flags four engine values and one console value sitting at exactly 128 bits with zero margin. Run the scope pass before re-verification. Proposed work, unallocated and by subject: the recovery-code widening plus the one test that pins three groups of five; the entropy-floor regression test with its mutation control; the policy-password draw increase, and if it is dropped, an explicit record that the verdict then depends on the score-the-default rule; recording the five zero-margin values; and separately, moving no verdict, a cross-language randomness inventory arm and a decision on whether the extension hardening tier should ride the merge gate. +**THE FIVE ZERO-MARGIN VALUES, named 2026-09-03 at `fd44b0f17`. The census reproduces from this repository alone, and it lands on exactly the shape the caveat above predicted: four engine values and one console value, all at 128 bits with zero margin.** An **AST** census over the gate's five walk roots finds **21 CSPRNG call sites** outside tests -- every `secrets.token_urlsafe` / `token_bytes` / `token_hex` / `randbits` and `os.urandom` call. AST rather than grep on purpose, and the difference is not cosmetic: the same pattern grepped returns 36 lines, because a docstring or comment NAMING a draw matches as readily as a draw. Resolving each named byte-count constant leaves six call sites at 16 bytes, which are **five distinct values** -- `dicomweb.py` draws the same boundary twice, once on the retry: + +| # | Site | Draw | What the value is | +|---|---|---|---| +| 1 | `messagefoundry/pipeline/sandbox.py:847` | `secrets.token_hex(16)` | the per-dispatch sandbox `request_id`, unpredictable ON PURPOSE so the code running dispatch N cannot pre-stage N+1's answer | +| 2 | `messagefoundry/uploads.py:497` | `secrets.token_hex(16)` | the uploaded-file `file_id`. Load-bearing rather than incidental: `api/app.py:3969-3970` states that what makes the by-id routes non-enumerable is the id itself, and it names the 128 bits | +| 3 | `messagefoundry/transports/dicomweb.py:329` (retry at `:333`) | `secrets.token_hex(16)` | the STOW-RS multipart boundary | +| 4 | `messagefoundry/transports/soap.py:288` | `os.urandom(16)` | the WS-Security nonce | +| 5 | `messagefoundry_webconsole/_security.py:205` -> `:324` | `secrets.token_urlsafe(16)` (`_NONCE_BYTES = 16`) | the console CSP nonce. The one console value, and the reason the count splits 4 + 1 | + +**Zero margin is not a finding on its own, and this row does not turn it into one.** Each of the five is AT the floor, which is a pass, not a shortfall -- the verb says "at least 128 bits". What makes them worth naming is exactly what `auth/service.py` demonstrated before PR #603: a value sitting ON a floor has nothing left to absorb a change made for an unrelated reason, and there the change was a rejection loop that a supported configuration pushed to 126.95 bits. So these five are where the next sub-floor value comes from, and the record now says which five rather than that there are five. **Two of the five are not obviously in the verb's reach at all, and that question belongs to the scope pass rather than to this row:** a STOW-RS multipart boundary is a delimiter whose requirement is non-collision with the payload, and a WS-Security nonce is a replay token whose requirement is uniqueness. Both are drawn at 128 bits anyway, so nothing turns on the answer today -- which is exactly why it is cheap to settle it before a re-score has to. **Three neighbours excluded, with the reason, because a bare count invites the wrong ones:** `store/crypto.py:704` and `store/backup_codec.py:181` draw 96 bits, but those are AES-GCM nonces, whose requirement is uniqueness under a per-key invocation bound (`store/gcm_bound.py`) rather than unguessability, so this verb does not reach them; `uploads.py:823` draws 32 bits for a temp-filename suffix inside an owner-only directory, a collision-avoidance value and not a secret. **And one value LEFT the list:** `auth/service.py:611` now draws `max(24, min_length)` -- 192 bits -- so the administrator credential the earlier paragraph measured at 126.95 bits under `password_require_symbol = true` is no longer on the floor at all. + +**THE HANDOFF -- two acts remain and NEITHER may be performed from an engine checkout.** Both belong to a Tracker seat working in the `MessageFoundry-vault` clone that sits beside this repository. Naming the requirement number here is fine; the cell's verdict, coverage and gaps are vaulted and stay there. + +1. **The ASVS 11.5.1 scorecard re-score -- this item's declared closing act.** The record is `docs/security/asvs-scorecard.toml` in the vault. `docs/security/` is gitignored here, so `git ls-files docs/security` from an engine checkout returns zero and the record looks absent rather than misplaced; that is the trap, not evidence. The status read needs no engine tree, corpus or network: `python scripts/asvs/scorecard.py --scorecard /docs/security/asvs-scorecard.toml --status`. A full verify additionally needs `--corpus` and an explicit `--root` naming the engine tree, and `verify` refuses a root that CONTAINS the scorecard. **The engine-side evidence the re-score needs is all on `main` and all citable:** `auth/totp.py:57-67` for the six-group shape and the comment that derives it; `tests/test_recovery_code_entropy.py` for the floor derived from the shipped constants plus its mutation control; `auth/service.py:600-614` for the 192-bit administrator draw; `ide/src/cspNonce.ts` for the 144-bit nonce; `ide/src/test/suite/extension-hardening.test.ts` for the extension's own negative test; and `scripts/security/crypto_inventory_check.py` (`check_non_python_randomness`) plus `tests/test_crypto_inventory_scanner.py` for the cross-language arm now riding a required context. **Anchor at a commit and print the ref pair:** no number that tool emits is a fact without the `# asvs-verify scorecard=X engine=Y` header beside it. +2. **The scope-completeness adversarial pass, which this item's own caveat says was never run.** The cheap-flip axis was reviewed once; the scope-completeness axis was not. The reason it matters is stated above: the census that supported the original verdict is a `secrets`-and-`os.urandom` pattern that can only find GOOD draws, so it cannot report a value drawn from something it does not pattern for. The five values in the table are what that census returns when it is asked the other question, and they are the starting point rather than the answer. **The pass should also re-ask the census question itself**, because the arm shipped here answers only the non-Python half of it: `ide/` is now inventoried for randomness, but PowerShell is not -- `scripts/**/*.ps1` is **56 files** (re-measured 2026-09-03, against a positive control of 53 of them matching `param(`; #1164 recorded 48 at its own commit, so the corpus grew) sitting inside `WALK_ROOTS[4]` that `discover()` never reads because it rglobs `*.py`, and #1164 records nine of them performing first-party crypto. That gap is filed under #1164, not here, and it is named so the completeness pass does not re-derive it as new. + ## 1173. research an honest pass for ASVS 11.5.2 -- an RNG-under-demand claim that survives the IDE extension staying in scope > 🔢 **Re-scored 2026-08-20 -> P3.** Value **3/10** · Difficulty **2/10** · _fill-in_. The specific surface the item said held the cell is gone: cspNonce.ts draws 144 bits from node:crypto and a shipped test pins that no production source uses Math.random for a nonce, so the scope-ruling temptation the item exists to refuse no longer has to be taken. The remainder is the item's evidence question plus a verdict edit in a record this repo does not hold (value 3, difficulty 2). _(was 3/10 · 5/10.)_ diff --git a/scripts/security/crypto_inventory_check.py b/scripts/security/crypto_inventory_check.py index 097aa1390..4076d17ea 100644 --- a/scripts/security/crypto_inventory_check.py +++ b/scripts/security/crypto_inventory_check.py @@ -44,6 +44,28 @@ registered here and the ``pipeline/`` call sites stay ``ssl``-free. That is centralization, not evasion: one place decides the TLS policy for every SMTP hop in the product. +**A SECOND ARM COVERS THE NON-PYTHON TREE (BACKLOG #1172, ASVS 11.5.1).** Everything above is an +``import ast`` walk of ``*.py``, so it is Python-only *by construction* and cannot see a randomness +draw in another language however the walk-set is spelled. :func:`check_non_python_randomness` scans +:data:`NON_PYTHON_WALK_ROOTS` (``ide/``, the shipped TypeScript VS Code extension) for randomness +sources and diffs them against :data:`NON_PYTHON_INVENTORY` the same bidirectional way. Three +properties are the point of it, and each is pinned by a test in +``tests/test_crypto_inventory_scanner.py``: + +* a **weak** source (:data:`WEAK_RANDOMNESS_PATTERNS`, i.e. ``Math.random()``) fails the gate and has + **no inventory row to hide behind** - registering one is refused on purpose, because ASVS 11.5.1 + states an entropy floor on the VALUE and #1172 names "narrow the declared scope so the hit stops + being visible" as a disqualified pass; +* a **strong** source is inventoried rather than merely tolerated, so a new CSPRNG site is reviewed; +* an **empty walk is a violation, not a clean result**. A scan that reaches no files, and an + inventory anchor that has gone missing, both red. The anchor (``ide/src/cspNonce.ts`` -> + ``randomBytes``) is what makes this instrument able to fail: break the walk and the stale direction + names it. + +This is a *randomness* inventory, and that is the whole claim it supports. The other first-party +crypto in ``ide/`` - the TLS floor ``ide/src/engineClient.ts`` applies to every https request - is +still not discoverable from here. + Stdlib only (no install), like ``scripts/security/scan_forbidden.py`` — runnable as a CI step and a pytest. Usage:: @@ -55,6 +77,7 @@ import argparse import ast +import re import sys from pathlib import Path @@ -72,11 +95,14 @@ # across the extension, and ``ide/src/engineClient.ts`` pins a TLS floor it applies to every https # request. Both are first-party crypto in a shipped artifact and NEITHER is discoverable from here. # -# ADDING ``ide/`` TO WALK_ROOTS WOULD BE A NO-OP THAT LOOKS LIKE A FIX: the Python AST scanner would -# find zero ``.py`` there, report clean, and the TypeScript sites would stay invisible while the tree -# gained a green whose greenness is evidence of nothing. Covering them needs a TypeScript arm on a -# merge-gating context, which is unbuilt. Until it exists this gate's green means "no undocumented -# crypto in the PYTHON of five roots", and that is the only claim it supports. +# ADDING ``ide/`` TO WALK_ROOTS WOULD STILL BE A NO-OP THAT LOOKS LIKE A FIX, and that has not +# changed: the Python AST scanner would find zero ``.py`` there, report clean, and the TypeScript +# sites would stay invisible while the tree gained a green whose greenness is evidence of nothing. +# ``ide/`` is covered by a SEPARATE arm instead (:data:`NON_PYTHON_WALK_ROOTS` below, BACKLOG #1172), +# which reads ``.ts``/``.js`` by pattern rather than by AST and rides this same required context. So +# this gate's green now means "no undocumented crypto in the PYTHON of five roots, AND no +# undocumented or weak RANDOMNESS source in the non-Python roots". The randomness half is the only +# claim the second arm supports - the extension's TLS floor is still outside both. # ``samples/`` is absent by design too, on the SAME rationale as the ``ide/`` exclusion (ASVS 11.1.3): # it is author-space EXAMPLE config, not shipped engine code, so its crypto (e.g. a content-fingerprint # ``hashlib.sha256`` in a sample Handler) is out of the deployed-system inventory scope — the gate @@ -86,6 +112,60 @@ # guards that consume the same tuple. WALK_ROOTS = ("messagefoundry", "messagefoundry_webconsole", "harness", "tee", "scripts") +# -------------------------------------------------------------------------------------------- +# The NON-PYTHON randomness arm (BACKLOG #1172, ASVS 11.5.1). +# -------------------------------------------------------------------------------------------- +#: First-party roots that hold shipped source in a language the AST walk above cannot read. Today +#: that is ``ide/``, the TypeScript VS Code extension. Kept separate from :data:`WALK_ROOTS` on +#: purpose: the two arms read different file types with different instruments, and merging them +#: would let one arm's green stand in for the other's silence. +NON_PYTHON_WALK_ROOTS = ("ide",) + +#: Suffixes the non-Python arm reads. Source only. +NON_PYTHON_SUFFIXES = (".ts", ".js", ".mjs", ".cjs") + +#: Never walked: dependency trees and bundler output. They are not first-party source, and a +#: node_modules walk would bury a real finding under thousands of vendor hits. +NON_PYTHON_SKIP_DIRS = frozenset({"node_modules", "out", "dist", ".vscode-test", "coverage"}) + +#: Randomness sources that are cryptographically sound. A file using one must carry a +#: :data:`NON_PYTHON_INVENTORY` row, so a new CSPRNG site is reviewed rather than merely allowed. +#: The lookbehind stops a longer identifier ending in the same word from matching. +STRONG_RANDOMNESS_PATTERNS: dict[str, re.Pattern[str]] = { + "randomBytes": re.compile(r"(? the STRONG sources it is +#: documented to draw from. Bidirectional like :data:`INVENTORY`, and the stale direction is what +#: makes this arm able to fail: if the walk ever breaks or the file moves, this row is unbacked and +#: the gate reds instead of reporting a clean empty scan. +NON_PYTHON_INVENTORY: dict[str, frozenset[str]] = { + # ADR 0065-adjacent, BACKLOG #1172. The single source of CSP nonces for every webview the + # extension builds: 18 bytes (144 bits) of ``randomBytes`` in base64url. A CSP nonce is a + # capability, so its entropy SOURCE is the security property; the file's own header explains why + # ``node:crypto`` and not ``crypto.getRandomValues`` (extension-host code, not webview code). + "ide/src/cspNonce.ts": frozenset({"randomBytes"}), +} + # Top-level stdlib modules that mean "crypto happens here". CRYPTO_MODULES = frozenset({"hashlib", "secrets", "hmac", "ssl", "argon2", "cryptography"}) @@ -504,17 +584,23 @@ def find_violations( inventory: dict[str, frozenset[str]], *, check_stale: bool, + noun: str = "crypto", + stale_verb: str = "imports", ) -> tuple[list[str], list[str]]: """Return ``(undocumented, stale)`` message lists. ``undocumented`` = a file uses a crypto module not recorded for it (the security-relevant direction). ``stale`` = an inventory entry the file no - longer backs (kept honest only when scanning the real package).""" + longer backs (kept honest only when scanning the real package). + + ``noun``/``stale_verb`` only reword the messages so the non-Python randomness arm can reuse this + diff verbatim rather than growing a second, silently different one. The defaults reproduce the + Python arm's wording byte-for-byte.""" undocumented: list[str] = [] for path, mods in sorted(actual.items()): extra = mods - inventory.get(path, frozenset()) if extra: documented = sorted(inventory.get(path, frozenset())) or "(file not in inventory)" undocumented.append( - f"{path}: undocumented crypto use {sorted(extra)} (documented: {documented})" + f"{path}: undocumented {noun} use {sorted(extra)} (documented: {documented})" ) stale: list[str] = [] if check_stale: @@ -522,15 +608,126 @@ def find_violations( gone = mods - actual.get(path, frozenset()) if gone: stale.append( - f"{path}: inventory lists {sorted(gone)} but the file no longer imports it" + f"{path}: inventory lists {sorted(gone)} but the file no longer {stale_verb} it" ) return undocumented, stale +def _is_comment_only(line: str) -> bool: + """True for a line that is nothing but a ``//`` or block-comment line. + + DELIBERATELY CONSERVATIVE, and the direction matters. A code line carrying a TRAILING comment is + still scanned in full, so a call cannot be hidden by putting a ``//`` earlier on the same line; + the cost is that a line beginning with ``*`` for some non-comment reason is skipped. The error + this can make is a false POSITIVE (flagging a call quoted inside a string), never a false + negative, which is the only safe direction for a security gate. Stripping comments properly would + need a JS lexer, and a naive strip would cut a line at the ``//`` of a URL and hide what follows. + + The shipped extension test ``ide/src/test/suite/extension-hardening.test.ts`` uses the same rule + for the same reason: ``cspNonce.ts`` NAMES ``Math.random`` in prose to explain why it is unusable. + """ + stripped = line.lstrip() + return stripped.startswith(("//", "*", "/*")) + + +def randomness_tokens_in(text: str) -> set[str]: + """The randomness-source tokens (strong and weak) a non-Python source draws from.""" + found: set[str] = set() + patterns = {**STRONG_RANDOMNESS_PATTERNS, **WEAK_RANDOMNESS_PATTERNS} + for line in text.splitlines(): + if _is_comment_only(line): + continue + for token, pattern in patterns.items(): + if token not in found and pattern.search(line): + found.add(token) + return found + + +def non_python_sources(root: Path) -> list[Path]: + """Every first-party non-Python source under ``root``, dependency and build trees pruned.""" + out: list[Path] = [] + for dirpath, dirnames, filenames in root.walk(): + dirnames[:] = [d for d in dirnames if d not in NON_PYTHON_SKIP_DIRS] + out.extend(dirpath / name for name in filenames if name.endswith(NON_PYTHON_SUFFIXES)) + return sorted(out) + + +def discover_non_python(root: Path, *, repo: Path) -> dict[str, frozenset[str]]: + """Map repo-relative path -> randomness tokens, for the files that draw randomness at all.""" + out: dict[str, frozenset[str]] = {} + for path in non_python_sources(root): + tokens = randomness_tokens_in(path.read_text(encoding="utf-8")) + if tokens: + out[path.relative_to(repo).as_posix()] = frozenset(tokens) + return out + + +def check_non_python_randomness(repo: Path) -> tuple[list[str], int]: + """Run the non-Python randomness arm. Returns ``(violation lines, files scanned)``. + + The file count is returned so the caller can PRINT what was scanned. An instrument that reports + "clean" without saying what it read is indistinguishable from one that read nothing, which is the + failure this arm exists to close for ASVS 11.5.1.""" + violations: list[str] = [] + actual: dict[str, frozenset[str]] = {} + scanned = 0 + + for name in NON_PYTHON_WALK_ROOTS: + root = repo / name + if not root.is_dir(): + violations.append( + f"randomness arm: declared walk root {name}/ is not a directory under {repo} - the " + "scan cannot report clean on a corpus it never reached" + ) + continue + files = non_python_sources(root) + if not files: + violations.append( + f"randomness arm: the walk over {name}/ reached ZERO " + f"{'/'.join(NON_PYTHON_SUFFIXES)} files, so a clean result would be VACUOUS. Either " + "the sources moved, the suffix list is wrong, or the pruned-directory list now eats " + "the corpus. Fix the walk; do not read this as a clean tree" + ) + continue + scanned += len(files) + actual |= discover_non_python(root, repo=repo) + + weak_tokens = set(WEAK_RANDOMNESS_PATTERNS) + for path, tokens in sorted(actual.items()): + weak = sorted(tokens & weak_tokens) + if weak: + violations.append( + f"{path}: WEAK randomness source {weak} in shipped non-Python source. There is no " + "inventory row for this and adding one is refused: ASVS 11.5.1 states an entropy " + "floor on the VALUE, and hiding the hit is the disqualified pass BACKLOG #1172 names." + " Draw from node:crypto instead - ide/src/cspNonce.ts is the worked example" + ) + + strong_actual = { + path: frozenset(tokens - weak_tokens) + for path, tokens in actual.items() + if tokens - weak_tokens + } + undocumented, stale = find_violations( + strong_actual, + NON_PYTHON_INVENTORY, + check_stale=True, + noun="randomness", + stale_verb="draws from", + ) + violations.extend(undocumented) + violations.extend(stale) + return violations, scanned + + def _assert_ide_is_typescript(repo: Path) -> list[str]: - """``ide/`` is excluded from the walk because it is TypeScript with zero ``.py`` files. Enforce + """``ide/`` is outside the PYTHON walk because it is TypeScript with zero ``.py`` files. Enforce that fact so a future ``.py`` there reds the gate (and forces a WALK_ROOTS + inventory update) - rather than silently escaping the crypto scan. Returns violation message lines (empty = OK).""" + rather than silently escaping the crypto scan. Returns violation message lines (empty = OK). + + Still needed after the non-Python arm landed, and for a different reason than before: that arm + reads ``.ts``/``.js``, so a ``.py`` under ``ide/`` would fall between the two walks rather than + merely outside one. This is the guard that catches it.""" ide = repo / "ide" if not ide.is_dir(): return [] @@ -560,6 +757,8 @@ def main(argv: list[str] | None = None) -> int: scanning_default = args.package is None actual: dict[str, frozenset[str]] = {} ide_violations: list[str] = [] + randomness_violations: list[str] = [] + randomness_scanned = 0 if scanning_default: repo = Path(__file__).resolve().parents[2] roots = [repo / name for name in WALK_ROOTS] @@ -570,6 +769,7 @@ def main(argv: list[str] | None = None) -> int: for root in roots: actual |= discover(root) ide_violations = _assert_ide_is_typescript(repo) + randomness_violations, randomness_scanned = check_non_python_randomness(repo) else: package = args.package if not package.is_dir(): @@ -591,10 +791,20 @@ def main(argv: list[str] | None = None) -> int: print("crypto-inventory: STALE inventory entries (remove them from INVENTORY):") for line in stale: print(f" - {line}") - if undocumented or stale or ide_violations: + if randomness_violations: + print("crypto-inventory: NON-PYTHON RANDOMNESS arm (ASVS 11.5.1, BACKLOG #1172) FAILED:") + for line in randomness_violations: + print(f" - {line}") + if undocumented or stale or ide_violations or randomness_violations: return 1 - print(f"crypto-inventory: OK - {len(actual)} documented crypto call site(s), no drift.") + # Print the corpus, not just the verdict: "clean" means nothing without what was read. + print( + f"crypto-inventory: OK - {len(actual)} documented crypto call site(s) across " + f"{len(WALK_ROOTS)} Python root(s), no drift; " + f"{randomness_scanned} non-Python source(s) scanned for randomness across " + f"{len(NON_PYTHON_WALK_ROOTS)} root(s), no weak source and no inventory drift." + ) return 0 diff --git a/tests/test_crypto_inventory_scanner.py b/tests/test_crypto_inventory_scanner.py index e159ae2e6..112d9ec00 100644 --- a/tests/test_crypto_inventory_scanner.py +++ b/tests/test_crypto_inventory_scanner.py @@ -15,6 +15,12 @@ * the walk-set is the pinned five roots (no ``samples/``); and * the ``ide/``-is-TypeScript exclusion is an **enforced** invariant, not a silent drop. +The last group (BACKLOG #1172, ASVS 11.5.1) freezes the **non-Python randomness arm** the Python +scanner structurally could not provide. Three properties, and a scan missing any one of them proves +nothing: it must FIND a planted ``Math.random()`` in a ``.ts`` file, it must NOT flag the legitimate +``randomBytes`` draw in ``ide/src/cspNonce.ts``, and it must REFUSE an empty walk rather than let one +render as clean. + PHI-free: it reads only code *names/paths*, never a secret value. The gate script is not an importable package (``scripts/`` has no ``__init__``), so load it standalone by file path like the doc guard does. """ @@ -130,3 +136,217 @@ def test_gate_is_clean_on_the_real_tree() -> None: # End-to-end: the widened walk over the five real roots matches the maintained inventory (no drift). gate = _gate() assert gate.main([]) == 0 + + +# --- BACKLOG #1172 / ASVS 11.5.1: the cross-language randomness arm ------------------------------- +# +# The Python arm is ``import ast`` over ``*.py``, so it cannot see a weak-PRNG draw in another +# language however its walk-set is spelled. These tests pin the arm that can. Each fixture repo is +# built on disk rather than mocked, because the property under test is what the WALK reaches. + +#: The real anchor, reproduced in miniature. Fixtures carry it by default so the arm's stale +#: direction (which fires when the inventory's one row is unbacked) does not drown the assertion +#: under test. The tests that want that direction ask for it explicitly. +_ANCHOR_REL = "ide/src/cspNonce.ts" +_ANCHOR_SRC = ( + 'import { randomBytes } from "node:crypto";\n' + 'export function nonce(): string {\n return randomBytes(18).toString("base64url");\n}\n' +) + + +def _ts_repo(tmp_path: Path, files: dict[str, str], *, with_anchor: bool = True) -> Path: + """Write ``files`` (repo-relative path -> text) into a throwaway repo root and return it.""" + written = dict(files) + if with_anchor: + written.setdefault(_ANCHOR_REL, _ANCHOR_SRC) + for rel, text in written.items(): + path = tmp_path / rel + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + return tmp_path + + +@pytest.mark.parametrize( + "call", + [ + "const n = Math.random();", + "const n = Math . random ();", # spacing must not hide it + "const n = window.Math.random();", # a qualified reference is the same draw + "const n = crypto.pseudoRandomBytes(18);", # node's own non-cryptographic sibling + ], +) +def test_a_planted_weak_draw_in_a_ts_file_is_flagged(tmp_path: Path, call: str) -> None: + # THE ARM'S REASON TO EXIST. Without this assertion its green is worth nothing: a scanner that + # cannot be made to fire is indistinguishable from one reading an empty corpus. + gate = _gate() + repo = _ts_repo( + tmp_path, {"ide/src/planted.ts": f"export function bad(): string {{\n {call}\n"} + ) + violations, scanned = gate.check_non_python_randomness(repo) + + assert scanned == 2, f"the fixture walk should have read both files, it read {scanned}" + weak = [v for v in violations if "WEAK randomness source" in v] + assert len(weak) == 1, violations + assert "ide/src/planted.ts" in weak[0], weak[0] + + +def test_a_weak_draw_in_a_js_webview_asset_is_flagged_too(tmp_path: Path) -> None: + # The webview scripts shipped as static assets rather than as .ts are the same kind of code and + # the same blind spot; a nonce minted there is as capability-granting as one minted in src/. + gate = _gate() + repo = _ts_repo(tmp_path, {"ide/media/panel.js": "const nonce = Math.random().toString(36);\n"}) + violations, _ = gate.check_non_python_randomness(repo) + assert any("ide/media/panel.js" in v and "WEAK" in v for v in violations), violations + + +def test_a_weak_draw_cannot_be_registered_away( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + # THE DISQUALIFIED MOVE, pinned. #1172 rules out narrowing the declared scope so a weak-PRNG hit + # stops being visible. An inventory row is exactly that move in miniature, so the weak check runs + # BEFORE the inventory diff and no row can silence it. Mutating the inventory here proves the + # ordering rather than asserting it: the gate must still red with the site fully "documented". + gate = _gate() + monkeypatch.setattr( + gate, + "NON_PYTHON_INVENTORY", + {"ide/src/planted.ts": frozenset({"Math.random", "randomBytes"})}, + ) + repo = _ts_repo( + tmp_path, + {"ide/src/planted.ts": "const n = Math.random();\n"}, + with_anchor=False, + ) + violations, _ = gate.check_non_python_randomness(repo) + assert any("WEAK randomness source" in v and "ide/src/planted.ts" in v for v in violations), ( + violations + ) + + +def test_no_weak_token_is_documentable_in_the_shipped_inventory() -> None: + # The structural half of the same property: the shipped inventory may not name a weak source. + gate = _gate() + documented = set().union(*gate.NON_PYTHON_INVENTORY.values()) + assert documented & set(gate.WEAK_RANDOMNESS_PATTERNS) == set(), documented + + +def test_the_real_cspnonce_draw_is_not_a_false_positive() -> None: + # THE OTHER HALF OF A USEFUL SCANNER. One that flags everything is as useless as one that flags + # nothing, and cspNonce.ts is the hard case on purpose: its header NAMES Math.random twice in + # prose to explain why it is unusable. The tokens must be exactly the sound draw it makes. + gate = _gate() + tokens = gate.randomness_tokens_in((_ROOT / _ANCHOR_REL).read_text(encoding="utf-8")) + assert tokens == {"randomBytes"}, tokens + + +def test_prose_naming_a_weak_source_is_not_a_hit() -> None: + gate = _gate() + prose = ( + "// WHY Math.random() cannot be used here: it is xorshift128+.\n" + " * A doc comment mentioning Math.random() is not a call.\n" + "/* Math.random() in a block comment is not a call either. */\n" + ) + assert gate.randomness_tokens_in(prose) == set() + + +def test_a_trailing_comment_cannot_hide_a_call() -> None: + # The conservative direction of _is_comment_only, asserted rather than assumed: only a WHOLLY + # commented line is skipped, so putting a // earlier on a code line does not launder the draw. + gate = _gate() + assert gate.randomness_tokens_in("const n = Math.random(); // harmless, honest\n") == { + "Math.random" + } + + +def test_an_undocumented_strong_source_is_flagged(tmp_path: Path) -> None: + # A sound draw is inventoried, not merely tolerated, so a NEW CSPRNG site is reviewed. This is + # the same bidirectional discipline the Python arm applies, reusing the same diff function. + gate = _gate() + repo = _ts_repo(tmp_path, {"ide/src/other.ts": "const k = randomBytes(32);\n"}) + violations, _ = gate.check_non_python_randomness(repo) + assert any( + "ide/src/other.ts" in v and "undocumented randomness use" in v for v in violations + ), violations + # ...and the anchor itself, which IS documented, must not be flagged in the same run. + assert not any(_ANCHOR_REL in v for v in violations), violations + + +def test_an_empty_walk_is_refused_rather_than_reported_clean(tmp_path: Path) -> None: + # THE NAMED FAILURE SHAPE. An empty scan and a clean scan must not look alike -- and this whole + # item exists because a gate could not see half its corpus while reporting green. + gate = _gate() + (tmp_path / "ide" / "src").mkdir(parents=True) + (tmp_path / "ide" / "README.md").write_text("no sources here\n", encoding="utf-8") + violations, scanned = gate.check_non_python_randomness(tmp_path) + assert scanned == 0 + assert any("ZERO" in v and "VACUOUS" in v for v in violations), violations + + +def test_a_missing_walk_root_is_refused(tmp_path: Path) -> None: + gate = _gate() + violations, scanned = gate.check_non_python_randomness(tmp_path) + assert scanned == 0 + assert any("is not a directory" in v for v in violations), violations + + +def test_a_broken_walk_reds_through_the_stale_anchor(tmp_path: Path) -> None: + # THE MUST-FIRE CONTROL, and the reason the inventory carries an anchor row at all. If the walk + # ever stops reaching cspNonce.ts -- a moved file, a wrong suffix list, an over-eager prune -- + # the row is unbacked and the arm reds instead of reporting a clean tree it never read. + gate = _gate() + repo = _ts_repo(tmp_path, {"ide/src/plain.ts": "export const x = 1;\n"}, with_anchor=False) + violations, scanned = gate.check_non_python_randomness(repo) + assert scanned == 1, "the walk must have run; this is not the empty-scan case" + assert any(_ANCHOR_REL in v and "no longer draws from" in v for v in violations), violations + + +def test_vendor_and_build_trees_are_pruned(tmp_path: Path) -> None: + # node_modules is not first-party source, and walking it would bury a real finding under vendor + # hits. Pruned here rather than filtered later so the walk stays fast on a developer checkout. + gate = _gate() + repo = _ts_repo( + tmp_path, + { + "ide/node_modules/pkg/index.js": "const n = Math.random();\n", + "ide/out/extension.js": "const n = Math.random();\n", + "ide/dist/extension.js": "const n = Math.random();\n", + }, + ) + violations, scanned = gate.check_non_python_randomness(repo) + assert scanned == 1, f"only the anchor is first-party source; scanned {scanned}" + assert violations == [], violations + + +def test_the_real_tree_passes_and_the_scan_is_non_trivial() -> None: + # END-TO-END on the shipped tree, with the corpus size asserted so a silently-emptied walk can + # never satisfy this test by returning no violations. + gate = _gate() + violations, scanned = gate.check_non_python_randomness(_ROOT) + assert violations == [], violations + assert scanned > 40, f"the extension's source corpus looks truncated: {scanned} file(s)" + + +def test_the_walk_reaches_both_halves_of_the_extension_corpus() -> None: + # Second positive control, for the half of the corpus that does not live under src/: the webview + # scripts shipped as static assets. Mirrors the shipped extension-hardening test's own control. + gate = _gate() + found = {p.relative_to(_ROOT).as_posix() for p in gate.non_python_sources(_ROOT / "ide")} + assert _ANCHOR_REL in found + assert "ide/media/stepsWebview.js" in found, sorted(found)[:20] + + +def test_the_shipped_anchor_is_both_discovered_and_inventoried() -> None: + # Both limbs on purpose (the shape #283 uses for the Python arm): the arm can SEE the site, and + # the site is accounted for. Either alone would pass while the other silently rotted. + gate = _gate() + discovered = gate.discover_non_python(_ROOT / "ide", repo=_ROOT) + assert discovered.get(_ANCHOR_REL) == frozenset({"randomBytes"}), discovered.get(_ANCHOR_REL) + assert _ANCHOR_REL in gate.NON_PYTHON_INVENTORY + + +def test_non_python_walk_roots_are_declared_and_disjoint_from_the_python_ones() -> None: + # The two arms read different file types with different instruments; merging the walk-sets would + # let one arm's green stand in for the other's silence. + gate = _gate() + assert gate.NON_PYTHON_WALK_ROOTS == ("ide",) + assert set(gate.NON_PYTHON_WALK_ROOTS).isdisjoint(gate.WALK_ROOTS)