diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index a19e6b606..c418f5488 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -8349,6 +8349,37 @@ filing. **RULED, as superseded:** The proposed corpus yields **5,350** entries clearing the shipped policy against the current **18**, at an unchanged floor. **Conditions:** it comes from the same MIT-licensed SecLists upstream the bundled file already cites, so this is a file swap from a vetted source rather than a new third-party dependency; `common_passwords.NOTICE` is updated to name the new list and its count; `password_min_length` is not touched; and **acceptance measures entries at or above `min_length`, never the total** -- a test asserting a total entry count passes today and proves nothing, which is exactly how the permissive reading came to look defensible. **Unchanged and still ruled out:** the hashed k-anonymity corpus named as the planned follow-up cannot satisfy the policy-matching clause, because `messagefoundry/auth/policy.py:76-80` stores 40-hex digests and an entry's length is therefore unknowable. That remains a separate subject. + +**AMENDED 2026-09-03. THE MEASUREMENT THAT SETTLES THE CELL, AND THE FALSE PREMISE IT RETIRES.** Briefs cut from the 2026-08-20 research still quote "18 of 10,000 bundled entries reach 15 characters". **That premise is false at HEAD and every downstream number built on it is wrong.** The union corpus landed before this PR. Measured on this tree through the shipped loader: + +| Measurement | Value | +|---|---| +| Bundled entries (lines / distinct) | 15,256 / 15,045 | +| Clearing `PasswordPolicy(check_breached=False, check_username=False)` at the shipped floor of 15 | **5,274** | +| ASVS 6.2.4 bar | at least 3,000 | + +**The cell is satisfied on the strict reading, with no code change and no corpus regeneration.** Rebuilding the corpus, which is what the brief asked for, would have been a no-op on a corpus that was already correct. + +**TWO INSTRUMENTS, EACH WITH A CONTROL THAT FIRES.** `awk` over raw lines gives 5,274 at 15-plus characters; `PasswordPolicy` through the loader gives 5,274 clearing the full policy. Controls: the file carries zero comment or blank lines, `length >= 1` returns the full 15,256, and the policy count moves with the floor (9,135 at 8, 6,020 at 10, 5,367 at 12, 3,173 at 16, 1,735 at 17, 512 at 20) rather than sitting constant. + +**THE TWO COUNTS AGREE EXACTLY, AND THAT IS A FINDING RATHER THAN A COINCIDENCE.** The deny-list is **not** a binding constraint: removing the context clause changes the count by 0, and zero of the 5,274 contain a `CONTEXT_WORDS` term -- **because the build filtered them out, not because the clause is inert**. Control: the clause rejects a planted probe containing `messagefoundry`. Adding a term would cut real entries (`qwerty` removes 98 of the 5,274, `love` 78, `2010` 30), so **any re-score trigger must name `password_check_context` beside `password_min_length`.** + +**HEADROOM IS ONE STEP, which nobody had measured.** The 3,000 bar still clears at a floor of 16 (3,173) and fails at 17 (1,735). Raising the shipped `password_min_length` past 16 is a corpus regeneration, not a settings edit. + +**THE SEVEN SHIPPED SITES THAT ASSERTED A STALE CORPUS SIZE, corrected in this PR.** The rebuild grew the list past ten thousand entries and seven places went on calling it a "top-10k" list. None is a control, so nothing broke and nothing reported it -- they simply asserted a number that had stopped being true, in the places an operator reads before deciding whether to configure a larger corpus. By symbol, not line number, because the line anchors in the older briefs have all drifted: the `messagefoundry.auth.policy` module docstring; the operator-facing startup warning in `_warn_if_corpus_unreadable` (`auth/service.py`); the `AuthSettings.password_breach_corpus_file` comment; the `password_check_breached` and `password_breach_corpus_file` rows of `docs/CONFIGURATION.md`; and two paragraphs of `docs/SECURITY.md` "Password policy". **The briefs named four; there were seven.** Each now describes the list without a size, so counts live only beside the data. The operator reference also gains the caveat that made the bundled list short in the first place: size a corpus by how many entries clear the policy, never by its line count. This half is cherry-picked from `1d012f8f9`, written for this item on 2026-08-22 and never merged. + +**TWO RESIDUALS THE COUNT MUST CARRY, or 5,274 reads as more than it is.** Both come from the 2026-08-20 research against the upstream million-entry member and are **not** re-measured here, because this PR did not fetch that file. **First, the slice is drawn from deep in the tail:** the first policy-clearing entry at the floor sits at global rank **778** and the three-thousandth at rank **660,561**. Frequency ordering that far down is close to meaningless, so "the top 3,000 which match the policy" is satisfied by construction rather than by these being the 3,000 likeliest guesses. **Second, roughly twelve percent of that depth are cracked-hash artefacts rather than human-chosen passwords** (353 of 3,000, measured at the slice's own depth), so effective human-relevant coverage is materially below the nominal number. + +**RE-SCORE BASIS FOR THE CELL (this PR does not touch the scorecard -- it lives in the vault clone).** ASVS **6.2.4** should move **partial -> pass**, on this evidence: the bundled corpus supplies **5,274** entries clearing the application's own `PasswordPolicy` at the shipped `password_min_length = 15`, against a bar of at least 3,000; the filter is the policy object rather than a length test, so the set is literally "which match the application's password policy". **Anchor the cell on the corpus-depth test rather than on a line of the corpus** -- a data-file anchor goes stale on the next regeneration. **Record both residuals above in the cell's reasoning**, and set the re-score trigger to `password_min_length` **and** `password_check_context`. The verdict no longer depends on the permissive reading the owner took on 2026-08-22: it holds on the strict reading too, so the reading question can be retired rather than re-argued. + +**THE CORPUS TOOLING IS A SEPARATE LANE, NOT DUPLICATED HERE.** By a 2026-09-03 scope split, the regeneration script, the corpus file, `common_passwords.NOTICE` and `tests/test_auth_core.py` -- including the corpus-depth gate this item has always wanted -- belong to a concurrent session. **This PR touches none of those four**, which is also why the seven prose corrections above ship without the gate that would keep them from rotting again; that gate exists in `1d012f8f9` and is worth taking from there. + +**THREE FINDINGS HANDED TO THAT LANE, measured here, so it does not rediscover them.** (1) **A recorded digest needs a `.gitattributes` pin or it is true only by luck.** `git hash-object` on the corpus returns `4482f231...` under `core.autocrlf=true` and `7ca7ef4a...` under `autocrlf=false` -- same file, same commit -- so a digest recorded without a pin accuses a clean checkout of tampering, and a wheel built on Windows ships CRLF matching no recorded checksum. The repository already solves this for its other digest-bearing vendored blob (`.github/actions/**/dist/** -text`, BACKLOG #1364). Converting the corpus working tree to LF produces an **empty content diff**, because git already stores LF, so the pin costs nothing. (2) **The by-floor counts above are worth gating, not just the headline** -- the headroom claim is derived from the table, so checking only the headline leaves the load-bearing half as unverified prose. (3) **A digest gate is a source-tree control, not an operator-side one.** `messagefoundry/integrity.py` attests installed files against the wheel's `RECORD`, but `_ATTESTED_SUFFIX` is `".py"`, so shipped **data** is skipped: an admin with venv-write and restart rights can truncate an installed `common_passwords.txt` to zero bytes, breach screening silently becomes a no-op, and the tripwire reports clean. That is pre-existing, is not a regression from this work, and needs its own item. + +**ONE FALSE PACKAGING CLAIM CORRECTED IN PASSING.** `pyproject.toml` named `tests/test_password_*.py` as the guard proving the corpus ships in the wheel. **No such file has ever existed**, so the stated control pointed at nothing while a working one -- `tests/test_auth_core.py` -- sat one name away. + +**STILL OPEN:** the scorecard re-score, and the corpus tooling in the other lane. The prose half is done. + ## 1135. research an honest pass for ASVS 6.2.11 -- whether publishing the enforced context words is durable enough to count > ๐Ÿ”ข **Re-scored 2026-08-20 -> P3.** Value **4/10** ยท Difficulty **2/10** ยท _fill-in_. The list a reader needs is published, so the remaining worth is the decay guard the item itself calls the load-bearing half plus the 6.2.11 pointer decision alongside 6.1.2. Difficulty 2: one small test asserting the doc list equals CONTEXT_WORDS, plus a docs pointer edit. _(was 4/10 ยท 3/10.)_ diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 7bfb54468..b60236bae 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -569,10 +569,10 @@ document ([SECURITY-DOCS-POLICY.md](SECURITY-DOCS-POLICY.md)). | `require_action_step_up` | bool | `true` | **action-bound step-up** ([ADR 0077](adr/0077-action-bound-step-up.md); ASVS 7.5.1/8.2.4). On by default: the durable-takeover JSON routes โ€” TOTP enroll/confirm and disable-MFA โ€” require a fresh proof **bound to that specific action** (`POST /me/reauth` with a matching `purpose`, single-use) instead of riding the session-wide `step_up_max_age_seconds` window. It closes the most-exploitable default: a session hijacked inside the 300 s login-seeded window could otherwise bind an attacker's authenticator with no fresh proof. It changes **only** those factor-binding routes โ€” the broad admin / replay / config / purge routes keep the session-window step-up. `false` reverts to the legacy session-window behaviour (0.2.x semantics), the documented org opt-out | | `password_min_length` | int | 15 | local-password policy โ€” ASVS 5.0-aligned, length-first | | `password_require_uppercase` / `password_require_lowercase` / `password_require_digit` / `password_require_symbol` | bool | `false` | character classes โ€” **opt-in**, each independently (ASVS 5.0 forbids mandatory composition); turn one on only for a legacy standard that still mandates it | -| `password_check_breached` | bool | `true` | reject known common/breached passwords against a bundled offline top-10k list (no live HIBP call) | +| `password_check_breached` | bool | `true` | reject known common/breached passwords against a bundled offline corpus (no live HIBP call). Its entry counts and the policy filter that built it are recorded once, in `common_passwords.NOTICE` beside the list. What makes an entry count at all is under `password_breach_corpus_file` below, and applies to the bundled list too | | `password_check_context` | bool | `true` | reject a local password that **contains** any deny-list term โ€” a case-insensitive substring test, anywhere in the value, not a whole-word or prefix match. The **twelve** terms are listed in full in [SECURITY.md](SECURITY.md) "Password policy"; an earlier revision of this row called them "app/vendor/HL7 terms" and gave four examples, which mis-stated the rule (five of the twelve are generic credential words unrelated to this application or to HL7). The list is fixed in code (`CONTEXT_WORDS` in [`auth/policy.py`](../messagefoundry/auth/policy.py)): **this flag turns the whole check on or off, and no setting adds or removes a term**, so a site needing its own vocabulary uses `password_breach_corpus_file` below โ€” which matches the *whole* password, never a substring | | `password_check_username` | bool | `true` | reject a password containing the user's **own username** (ASVS 6.2.11) | -| `password_breach_corpus_file` | path | โ€” | optional path to a **larger offline breach corpus** that augments the bundled top-10k list (ASVS 6.2.12): a plaintext list **or** an HIBP-style SHA-1 hash export (`HASH[:count]` lines, auto-detected). Fully offline โ€” still no live HIBP call. Use a curated subset, not the full ~40 GB HIBP set (it is loaded into memory). A path, not a secret | +| `password_breach_corpus_file` | path | โ€” | optional path to a **larger offline breach corpus** that augments the bundled one (ASVS 6.2.12): a plaintext list **or** an HIBP-style SHA-1 hash export (`HASH[:count]` lines, auto-detected). Fully offline โ€” still no live HIBP call. Use a curated subset, not the full ~40 GB HIBP set (it is loaded into memory). A path, not a secret. **Size this list by how many of its entries clear your policy, never by its line count**: an entry shorter than `password_min_length` can only reject a password the length rule already rejects, so it adds nothing. At the shipped minimum of 15 a general-purpose leaked-password list is mostly unreachable โ€” the bundled corpus was rebuilt for exactly this reason (BACKLOG #1134), and a hashed HIBP export cannot be filtered this way at all, because a SHA-1 digest does not carry the entry's length | | `lockout_threshold` | int | 5 | failed logins before lock (per account) | | `lockout_minutes` | int | 15 | lockout duration | | `bootstrap_expiry_hours` | int | 72 | the first-run bootstrap admin is auto-disabled once a second administrator exists, and โ€” while still unclaimed (never password-changed) โ€” this many hours after creation. `0` = no time expiry **of the ACCOUNT**, which is not no expiry of its **credential**: the printed first-run password is separately bounded by [`initial_password_expiry_hours`](#auth) ([BACKLOG #1245](BACKLOG.md)), so at `0` โ€” or at any value longer than that one โ€” the account survives while the credential still dies on the other clock. `bootstrap-admin.txt` states the **earlier** of the two | diff --git a/docs/SECURITY.md b/docs/SECURITY.md index 82d258bc9..eb9bc7e40 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -1487,8 +1487,8 @@ MFA step-up is now built (WP-14 native TOTP); a web console banner for the feed Local passwords follow an **ASVS 5.0-aligned** policy (WP-3): **min length 15**, **no mandatory character-class composition** (the `require_*` class flags are opt-in, default off โ€” ASVS forbids -mandatory composition), plus **offline breached/common-password screening** (a bundled top-10k list, -no live HIBP call) and a fixed **context-word deny-list**, enumerated in full below. Enforced +mandatory composition), plus **offline breached/common-password screening** (a bundled offline +corpus, no live HIBP call) and a fixed **context-word deny-list**, enumerated in full below. Enforced identically on create-user and change-password; tune via `[auth]` (see [CONFIGURATION.md](CONFIGURATION.md)). AD passwords are governed by Active Directory. @@ -1521,7 +1521,7 @@ Two further screens (ASVS 6.2.11 / 6.2.12), both on by default and fully offline user's own username (case-insensitive, for usernames โ‰ฅ 4 chars) is rejected, catching the common `jsmith2026`-style choice that the corpus can't. - **Larger operator breach corpus** (`password_breach_corpus_file`) โ€” point this at an offline list to - augment the bundled top-10k: a **plaintext** file *or* an **HIBP-style SHA-1-hash export** + augment the bundled corpus: a **plaintext** file *or* an **HIBP-style SHA-1-hash export** (`HASH[:count]` lines, auto-detected), checked locally with no network call. Use a curated subset (it's loaded into memory), not the full ~40 GB HIBP set; a configured-but-unreadable path is warned at startup and falls back to the bundled list. diff --git a/messagefoundry/auth/policy.py b/messagefoundry/auth/policy.py index 89b09dc3f..6efc34b95 100644 --- a/messagefoundry/auth/policy.py +++ b/messagefoundry/auth/policy.py @@ -8,8 +8,11 @@ rejection** (6.2.11). Defaults remain a direct improvement on Mirth, whose password requirements default to zero. Operators tune these via the ``[auth]`` settings section. -The breach corpus is a bundled offline top-10k common-password list (see ``data/common_passwords.txt`` -+ its ``.NOTICE``); the check is a case-insensitive set membership โ€” no network/live-HIBP call. +The breach corpus is a bundled offline common-password list (see ``data/common_passwords.txt`` and +its ``.NOTICE``, which carries the entry counts and the policy filter that built the list โ€” BACKLOG +#1134 grew it, so do not restate a size here); the check is a case-insensitive set membership โ€” no +network/live-HIBP call. Only entries at or above ``min_length`` add coverage: a shorter one can +reject only what the length clause already rejects. Operators can widen it with an offline ``breach_corpus_file`` (6.2.12) โ€” a plaintext list **or** an HIBP-style SHA-1-hash export (``HASH[:count]`` lines, auto-detected), still fully offline. (True HIBP k-anonymity needs a live range query, which this on-prem engine deliberately doesn't make.) diff --git a/messagefoundry/auth/service.py b/messagefoundry/auth/service.py index 6b284d856..3a116761c 100644 --- a/messagefoundry/auth/service.py +++ b/messagefoundry/auth/service.py @@ -82,7 +82,7 @@ def _warn_if_corpus_unreadable(path: str | None) -> None: except OSError as exc: _log.warning( "password_breach_corpus_file %r could not be read (%s); the larger breach corpus is " - "disabled (the bundled top-10k list still applies)", + "disabled (the bundled corpus still applies)", path, exc, ) diff --git a/messagefoundry/config/settings.py b/messagefoundry/config/settings.py index 0214873f9..ff6e67288 100644 --- a/messagefoundry/config/settings.py +++ b/messagefoundry/config/settings.py @@ -1865,9 +1865,10 @@ class AuthSettings(_Section): password_check_username: bool = ( True # reject passwords containing the user's own username (6.2.11) ) - # Optional path to a larger offline breach corpus that augments the bundled top-10k list (6.2.12): - # a plaintext list OR an HIBP-style SHA-1-hash export (HASH[:count] lines, auto-detected). Fully - # offline โ€” no live HIBP call. Use a curated subset, not the full ~40 GB HIBP set (loaded into memory). + # Optional path to a larger offline breach corpus that augments the bundled one (6.2.12): a + # plaintext list OR an HIBP-style SHA-1-hash export (HASH[:count] lines, auto-detected). Fully + # offline โ€” no live HIBP call. Use a curated subset, not the full ~40 GB HIBP set (loaded into + # memory). Only entries at or above password_min_length add coverage โ€” see docs/CONFIGURATION.md. password_breach_corpus_file: str | None = None lockout_threshold: int = 5 # consecutive failed logins before the account locks lockout_minutes: int = 15 diff --git a/pyproject.toml b/pyproject.toml index 0ce643456..8767f1087 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -223,8 +223,10 @@ messagefoundry-tray = "messagefoundry.tray.__main__:main" # The offline common-password screening corpus (messagefoundry/auth/data/, loaded via # importlib.resources) ships in the wheel automatically: hatchling packages every file under the # messagefoundry/ package, data included. (A previous force-include of that dir double-added the files -# and broke `build --wheel`.) tests/test_password_*.py load the corpus via importlib.resources, so a -# build-config change that dropped it would fail the suite โ€” that's the guard, not a force-include. +# and broke `build --wheel`.) The guard is tests/test_auth_core.py, whose #1134 corpus gates load the +# list via importlib.resources, so a build-config change that dropped it fails the suite -- not a +# force-include. (This named tests/test_password_*.py until 2026-09-03; no such file has ever +# existed, so the stated guard pointed at nothing while the real one sat one name away.) # --- CI TOOLCHAIN โ€” PEP 735 dependency groups (ADR 0034 ยง3) ---------------------------------------- #