diff --git a/docs/ASVS-L2-PHASE0-CHANGES.md b/docs/ASVS-L2-PHASE0-CHANGES.md index d5712f537..93a6d210c 100644 --- a/docs/ASVS-L2-PHASE0-CHANGES.md +++ b/docs/ASVS-L2-PHASE0-CHANGES.md @@ -236,13 +236,25 @@ erasure. ### Rotation schedule (ASVS 13.1.4 / 13.3.4) A rotation cadence per critical secret, justified against the threat model + HIPAA. These are -**operator-policy defaults** — the engine does **not** force-rotate or hard-expire a secret (rotation -*execution* stays operator- / secret-manager-driven, by design). It **does** now **monitor** the cadence +**operator-policy defaults** — the engine does **not** force-rotate a secret (rotation *execution* stays +operator- / secret-manager-driven, by design). **The store DEK is the one exception, and it is a hard +expiry**: see the note directly below. The engine **monitors** every cadence (ASVS 13.3.4, BACKLOG #282): the store DEK is tracked live-by-default and every configured secret class the engine holds is fingerprinted with a **DEK-derived keyed MAC** in store meta, so a **rotation is auto-detected** (the fingerprint changes → the clock resets) and a `secret_rotation_due` alert fires against the cadence below — never operator-attested, and carrying only dates + a one-way MAC, never a -value. Under `[security].enforcement=ENFORCE` a DEK past its max-age + grace **escalates** at restart. +value. + +> **The store DEK's calendar cadence is ENFORCED, not suggested** (ASVS 13.3.4, BACKLOG #1004). Under +> `[security].enforcement=ENFORCE` with a keyed store, a DEK past `store_key_max_age_days + +> enforce_grace_days` — or one whose age cannot be determined — **aborts engine start** +> (`StoreKeyRotationOverdueError`), in addition to the escalated alert rather than instead of it. The +> annual cadence in the table below is therefore a control on this one row, not a recommendation. The +> same key's **usage** axis has always refused unconditionally at 2^32 encrypts; this brings the calendar +> axis level with it. `[secret_rotation].enforce_store_key_expiry = false` keeps the alert and drops the +> refusal, and is reported by `security_loosenings()` on every boot. **This paragraph is a RECORD +> CORRECTION that FOLLOWS a shipped code change, not a lever:** the sentences it replaces became false +> when the refusal landed, and an edit that *substituted* for the code would be the forbidden move. | Secret (env var / connector setting) | Suggested cadence | Trigger / notes | |---|---|---| @@ -282,8 +294,9 @@ value. Under `[security].enforcement=ENFORCE` a DEK past its max-age + grace **e > cert (`[logging].forward_tls_client_cert`, a single combined PEM). Each secret **value** is > **`env()`-sourced — never the config file (the fixed `MEFOR_*` set is enforced by > `settings._FILE_SECRET_KEYS`) — and `/metadata` viewer-redacted**. Rotation *execution* stays operator- -> / secret-manager-driven — the engine never force-rotates or hard-expires a secret (session tokens are the -> one engine-expired credential) — but it now **monitors** the cadence and **auto-detects** a rotation +> / secret-manager-driven — the engine never force-rotates a secret, and the only credentials it +> hard-expires are session tokens and the **store DEK** (BACKLOG #1004: a calendar-overdue DEK refuses to +> start under ENFORCE) — but it now **monitors** the cadence and **auto-detects** a rotation > (ASVS 13.3.4 — see the rotation-watcher note below). **This enumeration is drift-guarded by > `tests/test_secret_rotation_inventory.py`**, which fails the build when a new `MEFOR_*` secret name (`…_TOKEN` / `…_SECRET` / `…_PASSWORD` / > `…_KEY`) appears in `messagefoundry/` without a row here — the exact 2026-07-16 regression that put this cell @@ -312,6 +325,13 @@ value. Under `[security].enforcement=ENFORCE` a DEK past its max-age + grace **e > - **ENFORCE escalation (committed).** Under `[security].enforcement=ENFORCE`, a DEK older than > `store_key_max_age_days + enforce_grace_days` escalates its `secret_rotation_due` alert (`enforced`, > logged at ERROR) at restart. +> - **ENFORCE refusal (BACKLOG #1004).** The same condition, on the same arithmetic, then **stops the +> start**: `enforce_store_key_expiry` raises `StoreKeyRotationOverdueError` out of `Engine.start()`, +> aborting the ASGI lifespan. It is called **outside** the blanket handler guarding the reconcile above +> — beneath it the refusal would be logged and stepped over, which is a traceback rather than a +> control. An **undetermined** age (the reconcile failed and no `store_key_last_rotated` override is +> set) refuses too: an undetermined age is not a young one. `enforce_store_key_expiry = false` keeps the +> alert, drops the refusal, and is named as a security loosening on every boot. > - **Review cadence.** This section + the keyed-secret enumeration are reviewed **quarterly** and on any > new `[store].cipher_provider` / `CRITICAL_SECRETS` change, tied to the drift guard above + > `scripts/security/crypto_inventory_check.py` so the definition cannot silently rot. diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index a19e6b606..a61729281 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -3814,6 +3814,12 @@ So the registered control for a context should record **what it does not break** ## 1004. ASVS 13.3.4 — the store DEK's calendar expiry alerts and never refuses; build the enforced stop with a loud opt-out +> 🚧 **ENGINE HALF BUILT 2026-09-03, DRAFT AND HELD — the vault half has not landed and this must not land alone.** The refusal ships as `enforce_store_key_expiry()` in `pipeline/secret_rotation.py`, called from `Engine.start` **outside** the blanket `except Exception` that guards the rotation-meta reconcile; an overdue calendar age and an **undetermined** age both refuse; `[secret_rotation].enforce_store_key_expiry` (default `true`) is the opt-out and `security_loosenings()` names it. Twelve engine-level and unit arms, plus a proven mutation control: moving the call inside that handler reds three tests, and the swallowed run logs *"secret-rotation stamp reconcile failed; continuing with config dates"* while the engine starts on an expired key. +> +> 🚧 **THE EXCEPTION NAME IS `StoreKeyRotationOverdueError`, and reporting it is the point.** This item rules that the name is a coordination-visible decision because the 13.3.4 absence claim is keyed on the exception NAME rather than on behaviour. `StoreKeyRotationOverdueError` carries **both** a rotation token and an overdue token, chosen deliberately over the more obvious `StoreKeyExpiredError` so that a name-keyed claim trips and the drift gate goes **red** — which this item calls the SAFE outcome, because it is the record noticing. **That is an intent, not a measurement:** `docs/security/` is gitignored here, so a builder in an engine checkout cannot read the token set and cannot confirm the claim actually fires. Whoever holds the vault half must check it rather than inherit this line. If it does **not** fire, the recorded absence has gone false silently and someone has to be told. +> +> 🚧 **The measuring-document edits are RECORD CORRECTIONS that FOLLOW the code, not the forbidden lever.** `docs/ASVS-L2-PHASE0-CHANGES.md`'s *"does not force-rotate or hard-expire a secret"* and its ENFORCE-escalation bullet became false the moment the refusal landed; leaving them standing would be the compensating-control-on-a-false-premise defect. They are corrected to describe shipped behaviour. `docs/CONFIGURATION.md`'s *"still an alert, never a refusal"* and a new `docs/SECURITY-LOOSENING.md` entry moved for the same reason. **No vault file was read or written.** + > 🔢 **Re-scored 2026-08-20 -> P1.** Value **8/10** · Difficulty **6/10** · _big bet_. Nothing shipped: the calendar axis in secret_rotation.py:319-351 still only alerts while crypto.py:688 refuses unconditionally on the usage axis, and enforce_store_key_expiry does not exist. Value 8 is now confirmed rather than assumed -- the 13.3.4 cell reads verdict=partial at level=3, so rung 8's first limb holds and the item's own conditional drop to 6 does not fire. Difficulty 6 because the refusal must be sited outside the blanket handler at engine.py:1062, the undetermined-age branch must refuse, the opt-out wires into security_loosenings (settings.py:4238), and the paired vault edit lands with it. _(was 8/10; difficulty was unscored.)_ > > **Filed 2026-08-04 — IN PROGRESS 2026-08-14. Scored 2026-08-04 → P1.** **Score of the item's own filing, SUPERSEDED by the re-score above:** Value **8/10** · Difficulty diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index 7bfb54468..3bb3ed94f 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -1189,10 +1189,17 @@ age** and raises the rotation-due alert (an [`[alerts]`](#alerts) event) when it `warn_days` of due. **Route it as `event_type = "secret_rotation"`** — that is the wire name the rule validator accepts; the longer `secret_rotation_due` is the internal `AlertSink` method name and is **rejected at config load** if you write it in a rule. It reads only the rotation -**dates** you configure here — **never any secret value** (PHI-free). This is a *reminder*, not -enforcement: it never rotates a key or blocks startup (run `rotate-key` to rotate the store DEK). Under -`[security].enforcement = enforce`, a store DEK past `store_key_max_age_days + enforce_grace_days` -escalates its alert at restart (`enforced = true`) — still an alert, never a refusal. +**dates** you configure here — **never any secret value** (PHI-free). It never *rotates* a key (run +`rotate-key` for that), and for every secret class except the store DEK it is a reminder only. + +**The store DEK's calendar expiry is ENFORCED** (ASVS 13.3.4, BACKLOG #1004). Under +`[security].enforcement = enforce` with a keyed store, a DEK past `store_key_max_age_days + +enforce_grace_days` escalates its alert at restart (`enforced = true`) **and refuses to start the +engine**. A DEK whose age cannot be determined — the rotation-meta reconcile failed and no +`store_key_last_rotated` is set — refuses on the same rule: an undetermined age is not a young one. This +matches the same key's **usage** ceiling, which has always refused unconditionally at 2^32 encrypts. Set +`enforce_store_key_expiry = false` to keep the alert and drop the refusal; that is a **security +loosening** and it is named on every boot and in `GET /security/posture`. The store DEK is tracked **live-by-default** (ASVS 13.3.4): at first keyed start the engine records a non-secret tracked-since stamp (the DEK key-id + first-seen date) in store meta and watches the DEK off @@ -1208,7 +1215,8 @@ tracked; set `warn_days = 0` to disable the reminder. | `store_key_last_rotated` | str | — | ISO `YYYY-MM-DD` the store DEK was last rotated; **unset ⇒ the DEK is still tracked live-by-default** off a persisted first-seen stamp (this date is an override) | | `store_key_max_age_days` | int | 365 | rotate the store DEK within this many days of its effective last-rotated (the operator date if set, else the persisted stamp) | | `secret_max_age_days` | int | 365 | max age for the **non-DEK** tracked secret classes (connector/AD/SMTP/Vault/OIDC), alerted this many days after their last observed fingerprint change | -| `enforce_grace_days` | int | 30 | under `[security].enforcement=enforce`, a DEK older than `store_key_max_age_days + this` escalates its rotation alert at restart (still an alert, never a refusal) | +| `enforce_grace_days` | int | 30 | under `[security].enforcement=enforce`, a DEK older than `store_key_max_age_days + this` escalates its rotation alert **and refuses to start** (see `enforce_store_key_expiry`) | +| `enforce_store_key_expiry` | bool | `true` | under `[security].enforcement=enforce`, a store DEK past `store_key_max_age_days + enforce_grace_days` — or one whose age cannot be determined — **aborts engine start**. `false` keeps the alert, drops the refusal, and is reported as a **security loosening** | ```toml [secret_rotation] diff --git a/docs/SECURITY-LOOSENING.md b/docs/SECURITY-LOOSENING.md index ea9fcec27..885adb9ff 100644 --- a/docs/SECURITY-LOOSENING.md +++ b/docs/SECURITY-LOOSENING.md @@ -65,15 +65,18 @@ section reference. | | `production_instance` | *derived from environment* | | Outside `[security]` | `[store].aad_bind` | `true` (at-rest values bound to their cell) | | | `[auth].ad_session_recheck_seconds` | `300` s (*conditional* — a loosening only once `ad_enabled`) | +| | `[secret_rotation].enforce_store_key_expiry` | `true` (a calendar-overdue store DEK refuses to start) | | Per-connection | `cleartext_accepted` | `false` on every outbound / `FhirLookup` (*connection-scoped* — see below) | | | `tls_allow_expired` | `false` on all six outbound connectors that take it (*connection-scoped*) | | | generic-ODBC `DATABASE` TLS | a verifying `odbc_params` keyword (*connection-scoped*; inbound **and** outbound) | -**Five of these do not live in `[security]`.** `[store].aad_bind` and `[auth].ad_session_recheck_seconds` -sit in their own sections for cohesion, and the last three are per-**connection** facts, not service +**Six of these do not live in `[security]`.** `[store].aad_bind`, +`[auth].ad_session_recheck_seconds` and `[secret_rotation].enforce_store_key_expiry` sit in their own +sections for cohesion, and the last three are per-**connection** facts, not service settings at all. They are listed and reported here anyway, because the rule is *one shipped posture, loosen only* — a deviation the registry cannot see is a second posture by the back door. The -first two are named by `security_loosenings()` from the loaded `[store]`/`[auth]` sections; the last +first three are named by `security_loosenings()` from the loaded +`[store]`/`[auth]`/`[secret_rotation]` sections; the last three are resolved from the loaded connection graph and passed in by name (see their entries below for exactly which surfaces see them, and which cannot). @@ -368,6 +371,26 @@ the call to the Console on 2026-09-02; the Console decided ([ADR 0118](adr/0118- `messagefoundry rotate-key` upgrades them `v1`→`v2` in place, so turning it back on does not strand an existing store. See [ADR 0019](adr/0019-pluggable-keyprovider-hsm-kms-vault.md) (2026-07-28 amendment). +### `[secret_rotation].enforce_store_key_expiry = false` — the store DEK's calendar expiry stops the engine no more +- **What you lose:** the **hard stop** on a calendar-expired data-encryption key. With it on, a DEK past + `store_key_max_age_days + enforce_grace_days` (365 + 30 as shipped) aborts engine start under + `[security].enforcement = enforce`, and so does a DEK whose age cannot be determined at all. With it + off, that same key keeps encrypting PHI at rest indefinitely and the only remaining signal is a + `secret_rotation_due` alert — which nobody has to answer. The engine documents an annual DEK cadence + (ASVS 13.3.4); this switch is what makes that cadence a control rather than a suggestion. +- **What you keep:** the alert. The opt-out suppresses the refusal and nothing else, deliberately — an + operator who accepted the risk still needs to be told the key is stale. The same key's **usage** axis + is also untouched: it still refuses unconditionally at 2^32 encrypts, with no opt-out at all. +- **When acceptable:** a scheduled maintenance start where rotating first is genuinely impossible, or a + restore/forensic bring-up against an old store whose key you must not rotate. Both are bounded windows. + Leaving it off permanently means the annual cadence is unenforced. +- **Compensating controls:** none that substitute. Route `event_type = "secret_rotation"` to a transport + somebody reads and treat `enforced = true` as an incident; the alert is the whole remaining signal. + `GET /security/posture` and the `serve` warning name the switch on every boot, so at least the gap is + visible. +- **Reversible:** yes, immediately — set it back to `true` (or delete the line) and restart. Nothing + about the key or the store changes either way; only whether the engine agrees to start. + ### `[auth].ad_session_recheck_seconds = 0` **with `ad_enabled`** — directory revocation stops propagating > **Conditional**, like `allowed_client_networks`. With no directory to reconcile against, `0` is not a > weaker choice — it is the only meaningful one — so it is reported as a deviation **only** when diff --git a/messagefoundry/__main__.py b/messagefoundry/__main__.py index 04475589d..d96599e18 100644 --- a/messagefoundry/__main__.py +++ b/messagefoundry/__main__.py @@ -1564,7 +1564,14 @@ def _serve(args: argparse.Namespace) -> int: # with its reason and an audit record; the #333 generic-ODBC TLS reminder naming the connection), # and completely by `messagefoundry check` and GET /security/posture, which both have the graph. _loosenings = security_loosenings( - settings.security, settings.store, settings.auth, settings.alerts, (), (), () + settings.security, + settings.store, + settings.auth, + settings.alerts, + settings.secret_rotation, + (), + (), + (), ) if _loosenings: _seclog = logging.getLogger(__name__) @@ -4973,6 +4980,7 @@ def _security(args: argparse.Namespace) -> int: from messagefoundry.config.settings import ( AlertsSettings, AuthSettings, + SecretRotationSettings, SecuritySettings, StoreSettings, load_settings, @@ -4988,6 +4996,9 @@ def _security(args: argparse.Namespace) -> int: # reporting a subset as if it were everything. _loosenings_partial = False _store, _auth, _alerts = StoreSettings(), AuthSettings(), AlertsSettings() + # BACKLOG #1004: [secret_rotation].enforce_store_key_expiry is a posture deviation too, so it is + # resolved from the same whole-file read and degrades with the same `loosenings_partial` marker. + _rotation = SecretRotationSettings() if Path(path).exists(): # An ABSENT file is not a degraded read — the shipped defaults ARE the effective posture there, # and `security show` is expected to work offline before any config exists. Only a file that @@ -4995,6 +5006,7 @@ def _security(args: argparse.Namespace) -> int: try: _full = load_settings(config_path=path) _store, _auth, _alerts = _full.store, _full.auth, _full.alerts + _rotation = _full.secret_rotation except (ValidationError, tomllib.TOMLDecodeError, OSError, ValueError): # The specific ways a settings file fails to resolve: a schema/cross-field violation, # malformed TOML, an unreadable path, and the plain ValueErrors load_settings raises for a @@ -5009,7 +5021,7 @@ def _loosenings(sec: SecuritySettings) -> list[dict[str, str]]: # posture. `messagefoundry check` and GET /security/posture are the complete surfaces. return [ {"switch": s, "risk": r} - for s, r in security_loosenings(sec, _store, _auth, _alerts, (), (), ()) + for s, r in security_loosenings(sec, _store, _auth, _alerts, _rotation, (), (), ()) ] #: Emitted alongside every loosening list this subcommand prints, so a reader can never mistake a diff --git a/messagefoundry/api/app.py b/messagefoundry/api/app.py index d419cafd9..9c323ef13 100644 --- a/messagefoundry/api/app.py +++ b/messagefoundry/api/app.py @@ -1586,6 +1586,11 @@ async def security_posture( # stash-or-default pattern — settings-scoped, so this route reports it completely even with no # graph loaded, unlike the connection-scoped cleartext_accepted set below. alerts_settings = getattr(request.app.state, "alerts_settings", None) or AlertsSettings() + # BACKLOG #1004: [secret_rotation] carries the store DEK's calendar-expiry opt-out. Same + # stash-or-default pattern — settings-scoped, so this route reports it completely with no graph. + secret_rotation_settings = ( + getattr(request.app.state, "secret_rotation_settings", None) or SecretRotationSettings() + ) # ADR 0153 + #333: the THREE connection-scoped deviations. Read LIVE off the running graph (so a # reload is reflected) — this route is where an operator learns a cleartext hop is being crossed # by declaration, an expired certificate is being honoured, or a generic DB hop has no verifying @@ -1616,6 +1621,7 @@ async def security_posture( store, auth_settings, alerts_settings, + secret_rotation_settings, cleartext_hops, expired_hops, db_hops, @@ -6143,6 +6149,10 @@ async def _alert_control(action: str, target: str) -> None: # window. None (the direct create_app / embedding path) leaves that check inert — deny-by-default # for a monitoring signal, and byte-identical to before. app.state.cert_monitor_settings = cert_monitor_settings + # BACKLOG #1004: back GET /security/posture's enforce_store_key_expiry loosening entry. + # None (direct create_app / embedding) leaves the route on shipped defaults, which report + # nothing — correct, because an app built without [secret_rotation] has not opted out. + app.state.secret_rotation_settings = secret_rotation_settings # #118: expose the connector SecretProvider so POST /alerts/test-email can resolve an # email_password_secret reference (fail-closed) exactly as notifier_from_settings does. None on # the embedded/test path — then only a plain env-sourced email_password can be tested. diff --git a/messagefoundry/config/settings.py b/messagefoundry/config/settings.py index 0214873f9..9070c4e55 100644 --- a/messagefoundry/config/settings.py +++ b/messagefoundry/config/settings.py @@ -3206,6 +3206,14 @@ class SecretRotationSettings(_Section): of due. It reads **only** the rotation *dates* an operator supplied here — never any secret value (PHI-free). Set ``warn_days`` to 0 to disable the reminder. + **The store DEK's calendar expiry is ENFORCED, not merely announced** (ASVS 13.3.4, BACKLOG #1004). + Under ``[security].enforcement=ENFORCE`` with a keyed store, a DEK past ``store_key_max_age_days + + enforce_grace_days`` — or one whose age cannot be determined at all — **aborts engine start** + (``StoreKeyRotationOverdueError``), alongside the escalated alert rather than instead of it. That + matches the same key's **usage** axis, which has always refused unconditionally at ``2**32`` + encrypts. ``enforce_store_key_expiry = false`` keeps the alert and drops the refusal; it is a + reported security loosening, not a quiet switch. + The store DEK is tracked **live-by-default** (ASVS 13.3.4, BACKLOG #282): at first keyed start the engine persists a non-secret tracked-since stamp (the DEK key-id + first-seen date) in store meta and watches the DEK off it, so setting ``store_key_last_rotated`` (an ISO ``YYYY-MM-DD`` date) is an @@ -3234,6 +3242,14 @@ class SecretRotationSettings(_Section): # ENFORCE escalation grace (ASVS 13.3.4): under [security].enforcement=ENFORCE, a DEK older than # store_key_max_age_days + this grace escalates its rotation alert (higher severity) at restart. enforce_grace_days: int = 30 + # ASVS 13.3.4 / BACKLOG #1004 — the calendar axis REFUSES, not just alerts. Under + # [security].enforcement=ENFORCE with a keyed store, a DEK past store_key_max_age_days + + # enforce_grace_days (or one whose age cannot be determined) aborts engine start. Default TRUE: + # the DEK's USAGE axis already refuses unconditionally at 2**32 encrypts, so a calendar axis + # shipping OFF would be strictly weaker than its own sibling on the same key, and a default-off + # build would buy the setting without the posture. Setting it false is a LOOSENING and + # security_loosenings() names it, so the opt-out is never silent. + enforce_store_key_expiry: bool = True @field_validator("warn_days") @classmethod @@ -4303,6 +4319,7 @@ def security_loosenings( store: StoreSettings, auth: AuthSettings, alerts: AlertsSettings, + secret_rotation: SecretRotationSettings, cleartext_hops: Sequence[str], expiry_relaxed_hops: Sequence[str], unverified_db_hops: Sequence[str], @@ -4315,7 +4332,8 @@ def security_loosenings( that iterates ``SecuritySettings.model_fields`` and fails on an unreported, unexempted one — plus an ENUMERATED set of deviations that live elsewhere: ``[store].aad_bind``, ``[auth].ad_session_recheck_seconds``, ``[alerts].email_use_tls``/``email_tls_verify`` (#323 - layer 3), and three per-connection deviations — ``cleartext_accepted``, ``tls_allow_expired``, and a + layer 3), ``[secret_rotation].enforce_store_key_expiry`` (#1004), and three per-connection + deviations — ``cleartext_accepted``, ``tls_allow_expired``, and a generic-ODBC ``DATABASE`` hop with TLS unenforced (#333). It is NOT yet an exhaustive registry of every security-relevant switch in every section; ``[store]``/``[auth]`` carry others (``encrypt``, ``trust_server_certificate``, ``enabled``, ``require_mfa``, @@ -4521,6 +4539,20 @@ def security_loosenings( "between cells decrypts instead of failing its auth tag (no effect without a store key)", ) ) + # BACKLOG #1004 (ASVS 13.3.4). Stated as what the SITE gives up rather than "a setting is off": the + # engine keeps starting on a key past its documented cadence, and the only remaining signal is an + # alert nobody has to answer. Named here because a silent opt-out from a refusal is indistinguishable + # from the refusal never having been built — which is the defect the refusal replaced. + if not secret_rotation.enforce_store_key_expiry: + out.append( + ( + "enforce_store_key_expiry", + "the store data-encryption key's CALENDAR expiry does not stop anything — a DEK past " + "its max age plus grace, or one whose age cannot be determined, still starts the " + "engine and keeps encrypting PHI at rest, with an alert as the only signal (the same " + "key's 2**32-encrypt usage ceiling still refuses unconditionally)", + ) + ) # Conditional on ad_enabled, like allowed_client_networks above: with no directory there is nothing to # reconcile against, so 0 is not a weaker choice, it is the only meaningful one. if auth.ad_enabled and not auth.ad_session_recheck_seconds: diff --git a/messagefoundry/pipeline/engine.py b/messagefoundry/pipeline/engine.py index 93731b2cc..cd0319d7d 100644 --- a/messagefoundry/pipeline/engine.py +++ b/messagefoundry/pipeline/engine.py @@ -68,6 +68,7 @@ MonitoredSecret, SecretRotationRunner, SecretStamp, + enforce_store_key_expiry, reconcile_rotation_meta, secrets_from_settings_and_stamps, ) @@ -1065,6 +1066,26 @@ async def start(self) -> None: log.exception( "secret-rotation stamp reconcile failed; continuing with config dates" ) + # ASVS 13.3.4 / BACKLOG #1004 — the DEK's CALENDAR expiry refusal, and its placement is + # the control. It sits OUTSIDE the `except Exception` directly above ON PURPOSE: that + # handler's entire body is a log call, so a refusal raised beneath it would be logged + # and stepped over, and the engine would start normally on an expired key. A gate a + # handler swallows is a traceback, not a control. From here it propagates out of + # Engine.start() and aborts the ASGI lifespan. + # + # It also covers the SECOND-ORDER swallow: when the reconcile above fails, the stamps + # stay empty, so a gate written to read them would silently not fire. An undetermined + # age REFUSES rather than reading as a young one (the item's ruling). `cipher_info()` + # is re-read rather than hoisted so the reconcile's own guarded call keeps its + # behaviour; if it were to fail here the engine refuses to start, which is the + # fail-closed direction. + enforce_store_key_expiry( + self._secret_rotation_settings, + self._secret_rotation_stamps, + enforcement=self._security_enforcement, + dek_key_id=self.store.cipher_info().active_key_id, + alert_sink=self._alert_sink, + ) self._secret_rotation_runner = SecretRotationRunner( self._tracked_secrets, self._secret_rotation_settings, diff --git a/messagefoundry/pipeline/secret_rotation.py b/messagefoundry/pipeline/secret_rotation.py index 00deaa5c7..031a69c74 100644 --- a/messagefoundry/pipeline/secret_rotation.py +++ b/messagefoundry/pipeline/secret_rotation.py @@ -15,7 +15,10 @@ class the engine holds is fingerprinted with a **DEK-derived KEYED MAC** (HMAC — never a salted hash, so a low-entropy secret is not offline-guessable) into store meta; when a fingerprint changes the clock is reset (rotation **auto-detected**, never operator-attested). Under ``[security].enforcement=ENFORCE`` a -DEK past ``max_age + grace`` escalates its alert severity at restart. +DEK past ``max_age + grace`` escalates its alert severity at restart **and the engine refuses to start** +(:func:`enforce_store_key_expiry`, BACKLOG #1004) — the calendar axis now stops, the way the usage axis +always has. ``[secret_rotation].enforce_store_key_expiry = false`` is the operator escape, and it is a +named security loosening rather than a quiet one. **PHI/secret-safe:** a secret value is read only transiently to compute its keyed MAC; only the MAC + the *dates* (plus a static human label + config identifier per secret) are ever persisted, alerted, or @@ -56,6 +59,8 @@ class the engine holds is fingerprinted with a **DEK-derived KEYED MAC** (HMAC "SecretCheck", "SecretRotationRunner", "SecretStamp", + "StoreKeyRotationOverdueError", + "enforce_store_key_expiry", "reconcile_rotation_meta", "secrets_from_settings", "secrets_from_settings_and_stamps", @@ -351,6 +356,155 @@ def _maybe_escalate_dek( log.warning("secret_rotation ENFORCE escalation sink failed", exc_info=True) +class StoreKeyRotationOverdueError(RuntimeError): + """The store DEK's **calendar** expiry is past due — or its age cannot be determined at all — under + ``[security].enforcement=ENFORCE``, so the engine REFUSES to start (ASVS 13.3.4, BACKLOG #1004). + + The DEK has two expiry axes and, before this, only one of them stopped anything. The **usage** axis + refuses unconditionally at ``2**32`` encrypts + (:class:`~messagefoundry.store.crypto.AesGcmCipher` raises ``CipherError``); the **calendar** axis + computed the same overdue condition and emitted a single alert. An alert an operator can miss is not + an expiry, so the documented annual cadence was unenforced. This is the calendar axis's refusal. + + **Deliberately raised OUTSIDE the blanket ``except Exception`` that guards the rotation-meta + reconcile in :meth:`Engine.start`.** Sited beneath it, this exception would be logged and stepped + over and the engine would start normally — a traceback, not a control. It propagates out of + ``Engine.start()`` and aborts the ASGI lifespan. + + **Two distinct causes, and both refuse.** A *known* effective last-rotated date past + ``store_key_max_age_days + enforce_grace_days``; or an **undetermined** age (the reconcile failed, + leaving no stamp, and no ``store_key_last_rotated`` override is set). An undetermined age is not a + young one: rendering it as safe would re-create the very defect this class replaces, one level + removed and on shipped defaults. + + The operator escape is ``[secret_rotation].enforce_store_key_expiry = false``, which suppresses the + refusal and keeps the alert. It is a LOOSENING and ``security_loosenings()`` names it on every + boot.""" + + def __init__(self, detail: str, *, last_rotated: str, days_overdue: int | None) -> None: + super().__init__( + f"{_DEK_LABEL} ({_DEK_SECRET_ID}): {detail}. The engine REFUSES to start under " + "[security].enforcement=enforce. Rotate the key (`messagefoundry rotate-key`), or record " + "the real rotation date in [secret_rotation].store_key_last_rotated, or set " + "[secret_rotation].enforce_store_key_expiry = false to run on a calendar-expired key " + "(a LOOSENING — it is named in the boot posture read-out)." + ) + self.last_rotated = last_rotated + self.days_overdue = days_overdue + + +def enforce_store_key_expiry( + settings: SecretRotationSettings, + stamps: Mapping[str, SecretStamp], + *, + enforcement: SecurityEnforcement, + dek_key_id: str | None, + alert_sink: AlertSink | None = None, + now: float | None = None, +) -> None: + """The store DEK's calendar-expiry **refusal** (ASVS 13.3.4, BACKLOG #1004) — raise + :class:`StoreKeyRotationOverdueError` when the key is past + ``store_key_max_age_days + enforce_grace_days``, or when its age cannot be determined. + + **Call this OUTSIDE the caller's blanket exception handler.** It is a control, and a control a + handler swallows is a log line. :meth:`Engine.start` calls it after the guarded reconcile precisely + so the refusal reaches the lifespan. + + Gated exactly as the escalation alert beside it is: ``[security].enforcement=ENFORCE`` and a keyed + store (``dek_key_id`` present — a keyless or ``vault_transit`` store has no local DEK to expire). It + reuses the SAME arithmetic ``_maybe_escalate_dek`` computes, off the same three shipped knobs, so + the alert and the refusal can never disagree about what "overdue" means. + + ``enforce_store_key_expiry = false`` suppresses the **raise** and nothing else: the reminder still + fires, because an operator who accepted the risk still needs to be told the key is stale. + + PHI/secret-safe: reads dates and a one-way key-id, never key material.""" + if enforcement is not SecurityEnforcement.ENFORCE: + return + if not dek_key_id: + return # keyless / vault_transit store — no local DEK, nothing to expire + sink = alert_sink or LoggingAlertSink() + ts = time.time() if now is None else now + today = datetime.datetime.fromtimestamp(ts, tz=_UTC).date() + + # An EMPTY stamp map means the reconcile did not complete (the engine assigns the stamps only on + # its success path), so `_maybe_escalate_dek` — which runs at the END of a successful reconcile — + # never fired and NOTHING has alerted about this key yet. That is the flag the alerting below reads, + # and it is why an alert here is not a duplicate: on the normal path the escalation alert has + # already gone out on this exact condition, and sending a second one would train operators to + # ignore it. + reconcile_alerted = bool(stamps) + + def _alert(last_rotated: str, days_overdue: int) -> None: + """Emit the enforced rotation alert, CONTAINED. A broken notifier must never swallow the stop, + so this is called before the raise and its failure is logged rather than propagated.""" + try: + sink.secret_rotation_due( + _DEK_LABEL, + secret=_DEK_SECRET_ID, # nosec B106 — the secret's env-var NAME, never its value + last_rotated=last_rotated, + days_overdue=days_overdue, + enforced=True, + ) + except Exception: + log.warning("secret_rotation expiry-refusal sink failed", exc_info=True) + + dek_stamp = stamps.get(_DEK_SECRET_ID) + if settings.store_key_last_rotated: + eff_last = datetime.date.fromisoformat(settings.store_key_last_rotated) + elif dek_stamp is not None: + eff_last = dek_stamp.last_rotated + else: + # UNDETERMINED age, and the store IS keyed and DOES track rotation meta — so this is not the + # keyless case, it is a reconcile that failed and was swallowed upstream. Ruled to REFUSE: an + # undetermined age is not a young one, and the empty-scan-reads-as-clean-scan failure is exactly + # what this item exists to remove. The alert is emitted IN ADDITION, never instead. + # + # `days_overdue` here is NOT a measurement — there is no date to measure from. It is the + # DECISION expressed in the field's own units: the smallest value satisfying the refusal + # predicate, so a rule keyed on `days_overdue` routes it like any other enforced expiry. The + # truth is carried by `last_rotated="unknown"`, which every sink renders. + _alert("unknown", settings.enforce_grace_days + 1) + if not settings.enforce_store_key_expiry: + log.error( + "store DEK rotation age is UNDETERMINED and the calendar expiry is DISABLED " + "([secret_rotation].enforce_store_key_expiry=false) — starting on an unverified key age" + ) + return + raise StoreKeyRotationOverdueError( + "its rotation age could not be determined (the rotation-meta reconcile failed and no " + "[secret_rotation].store_key_last_rotated is set), and an undetermined age is not a " + "young one", + last_rotated="unknown", + days_overdue=None, + ) + + days_overdue = (today - eff_last).days - settings.store_key_max_age_days + if days_overdue <= settings.enforce_grace_days: + return + if not reconcile_alerted: + # The narrow path where the overdue branch is reached with NOTHING having alerted: the reconcile + # failed AND the operator set `store_key_last_rotated`, so there is an effective date to judge + # but no escalation alert went out. Without this the opt-out would start on a knowably expired + # key in silence — which is the exact shape this item exists to remove. + _alert(eff_last.isoformat(), days_overdue) + if not settings.enforce_store_key_expiry: + log.error( + "store DEK is %d day(s) past its max age (last_rotated=%s) and the calendar expiry is " + "DISABLED ([secret_rotation].enforce_store_key_expiry=false) — starting on an expired key", + days_overdue, + eff_last.isoformat(), + ) + return + raise StoreKeyRotationOverdueError( + f"it is {days_overdue} day(s) past its {settings.store_key_max_age_days}-day max age " + f"(last rotated {eff_last.isoformat()}), beyond the " + f"{settings.enforce_grace_days}-day enforcement grace", + last_rotated=eff_last.isoformat(), + days_overdue=days_overdue, + ) + + class SecretRotationRunner: """Periodically scans the tracked secrets and raises ``secret_rotation_due`` alerts for any overdue or within-window. Construct with a ``secret_source`` callable (recomputed each pass) + the diff --git a/tests/test_alert_smtp_tls.py b/tests/test_alert_smtp_tls.py index 49d582102..785cc621f 100644 --- a/tests/test_alert_smtp_tls.py +++ b/tests/test_alert_smtp_tls.py @@ -29,6 +29,7 @@ INSECURE_TLS_ESCAPE_ENV, AlertsSettings, AuthSettings, + SecretRotationSettings, SecuritySettings, StoreSettings, security_loosenings, @@ -202,7 +203,10 @@ def _names(**kw: Any) -> list[str]: ) sec = SecuritySettings(**kw.pop("security", {})) return [ - n for n, _ in security_loosenings(sec, StoreSettings(), AuthSettings(), alerts, (), (), ()) + n + for n, _ in security_loosenings( + sec, StoreSettings(), AuthSettings(), alerts, SecretRotationSettings(), (), (), () + ) ] @@ -238,7 +242,14 @@ def test_an_unconfigured_alert_transport_reports_no_hop_deviation() -> None: names = [ n for n, _ in security_loosenings( - SecuritySettings(), StoreSettings(), AuthSettings(), bare, (), (), () + SecuritySettings(), + StoreSettings(), + AuthSettings(), + bare, + SecretRotationSettings(), + (), + (), + (), ) ] assert "email_use_tls" not in names diff --git a/tests/test_client_network_allowlist.py b/tests/test_client_network_allowlist.py index 8cd47d63b..9cc5a6ca5 100644 --- a/tests/test_client_network_allowlist.py +++ b/tests/test_client_network_allowlist.py @@ -34,6 +34,7 @@ AlertsSettings, ApiSettings, AuthSettings, + SecretRotationSettings, SecuritySettings, ServiceSettings, StoreSettings, @@ -50,7 +51,9 @@ def _loosenings(sec: SecuritySettings) -> list[tuple[str, str]]: The registry takes all four inputs as REQUIRED arguments deliberately (ADR 0148: one posture, and a deviation the registry cannot see is a second posture by the back door). The tests below are about the ``[security]`` switches specifically, so the other three are pinned at shipped values here.""" - return security_loosenings(sec, StoreSettings(), AuthSettings(), AlertsSettings(), (), (), ()) + return security_loosenings( + sec, StoreSettings(), AuthSettings(), AlertsSettings(), SecretRotationSettings(), (), (), () + ) PW = "a-strong-test-passphrase" # >=15, no app/vendor terms — satisfies the ASVS policy diff --git a/tests/test_memory_encryption_readout.py b/tests/test_memory_encryption_readout.py index 32cffbfec..17f0523fa 100644 --- a/tests/test_memory_encryption_readout.py +++ b/tests/test_memory_encryption_readout.py @@ -44,6 +44,7 @@ from messagefoundry.config.settings import ( AlertsSettings, AuthSettings, + SecretRotationSettings, SecuritySettings, StoreSettings, load_settings, @@ -57,7 +58,9 @@ def _loosenings(sec: SecuritySettings) -> list[tuple[str, str]]: The registry takes all four inputs as REQUIRED arguments deliberately (ADR 0148: one posture, and a deviation the registry cannot see is a second posture by the back door). The tests below are about the ``[security]`` switches specifically, so the other three are pinned at shipped values here.""" - return security_loosenings(sec, StoreSettings(), AuthSettings(), AlertsSettings(), (), (), ()) + return security_loosenings( + sec, StoreSettings(), AuthSettings(), AlertsSettings(), SecretRotationSettings(), (), (), () + ) SAMPLES_CONFIG = Path(__file__).resolve().parents[1] / "samples" / "config" diff --git a/tests/test_security_config.py b/tests/test_security_config.py index 9ad66726b..e991350c7 100644 --- a/tests/test_security_config.py +++ b/tests/test_security_config.py @@ -20,6 +20,7 @@ from messagefoundry.config.settings import ( AlertsSettings, AuthSettings, + SecretRotationSettings, SecuritySettings, ServiceSettings, StoreSettings, @@ -34,7 +35,9 @@ def _loosenings(sec: SecuritySettings) -> list[tuple[str, str]]: The registry takes all four inputs as REQUIRED arguments deliberately (ADR 0148: one posture, and a deviation the registry cannot see is a second posture by the back door). The tests below are about the ``[security]`` switches specifically, so the other three are pinned at shipped values here.""" - return security_loosenings(sec, StoreSettings(), AuthSettings(), AlertsSettings(), (), (), ()) + return security_loosenings( + sec, StoreSettings(), AuthSettings(), AlertsSettings(), SecretRotationSettings(), (), (), () + ) SAMPLES_CONFIG = Path(__file__).resolve().parents[1] / "samples" / "config" diff --git a/tests/test_security_posture_defaults.py b/tests/test_security_posture_defaults.py index 5a17beaee..ff4b2c8af 100644 --- a/tests/test_security_posture_defaults.py +++ b/tests/test_security_posture_defaults.py @@ -28,6 +28,7 @@ from messagefoundry.config.settings import ( AlertsSettings, AuthSettings, + SecretRotationSettings, SecuritySettings, ServiceSettings, StoreSettings, @@ -55,6 +56,7 @@ def _names( store: StoreSettings | None = None, auth: AuthSettings | None = None, alerts: AlertsSettings | None = None, + rotation: SecretRotationSettings | None = None, cleartext_hops: tuple[str, ...] = (), expiry_hops: tuple[str, ...] = (), db_hops: tuple[str, ...] = (), @@ -67,6 +69,7 @@ def _names( store or StoreSettings(), auth or AuthSettings(), alerts or AlertsSettings(), + rotation or SecretRotationSettings(), cleartext_hops, expiry_hops, db_hops, @@ -100,6 +103,7 @@ def test_aad_bind_off_is_a_named_loosening() -> None: StoreSettings(aad_bind=False), AuthSettings(), AlertsSettings(), + SecretRotationSettings(), (), (), (), @@ -122,6 +126,7 @@ def test_aad_bind_loosening_names_its_no_op_caveat() -> None: StoreSettings(aad_bind=False), AuthSettings(), AlertsSettings(), + SecretRotationSettings(), (), (), (), @@ -136,7 +141,16 @@ def test_aad_bind_loosening_names_its_no_op_caveat() -> None: def test_recheck_zero_with_ad_enabled_is_a_named_loosening() -> None: auth = _ad(ad_session_recheck_seconds=0) named = dict( - security_loosenings(SecuritySettings(), StoreSettings(), auth, AlertsSettings(), (), (), ()) + security_loosenings( + SecuritySettings(), + StoreSettings(), + auth, + AlertsSettings(), + SecretRotationSettings(), + (), + (), + (), + ) ) assert "ad_session_recheck_seconds" in named assert "revocation" in named["ad_session_recheck_seconds"] @@ -424,6 +438,7 @@ def test_cleartext_accepted_is_a_named_loosening() -> None: StoreSettings(), AuthSettings(), AlertsSettings(), + SecretRotationSettings(), ("OB_LEGACY", "OB_LAB"), (), (), @@ -457,6 +472,7 @@ def test_expiry_relaxation_is_a_named_loosening() -> None: StoreSettings(), AuthSettings(), AlertsSettings(), + SecretRotationSettings(), (), ("OB_PARTNER_ADT", "OB_LAB_ORU"), (), @@ -482,6 +498,7 @@ def test_generic_odbc_unenforced_tls_is_a_named_loosening() -> None: StoreSettings(), AuthSettings(), AlertsSettings(), + SecretRotationSettings(), (), (), ("OB_PG_RESULTS", "inbound:IB_PG_ORDERS"), diff --git a/tests/test_store_key_calendar_expiry.py b/tests/test_store_key_calendar_expiry.py new file mode 100644 index 000000000..166e762bc --- /dev/null +++ b/tests/test_store_key_calendar_expiry.py @@ -0,0 +1,435 @@ +# SPDX-License-Identifier: AGPL-3.0-or-later +# Copyright (C) 2026 MessageFoundry Organization and contributors +"""ASVS 13.3.4 / BACKLOG #1004 — the store DEK's CALENDAR expiry now REFUSES, with a loud opt-out. + +The key has two expiry axes and, before this, only one of them stopped anything. The **usage** axis +refuses unconditionally at ``2**32`` encrypts (``AesGcmCipher._count_invocation``, covered by +``tests/test_asvs_gcm_invocation_bound.py``). The **calendar** axis computed the same overdue condition +and emitted a single alert inside a try/except that logged a *sink* failure — no raise, no exit. + +Five arms, and the fifth is the one that separates a control from a log line: + +1. an overdue calendar age REFUSES (unit, and again through ``Engine.start()``); +2. an UNDETERMINED age REFUSES — not "alerts", and not "reads as young"; +3. ``enforce_store_key_expiry = false`` disables the refusal AND is named by ``security_loosenings()``; +4. a non-overdue key is unaffected (without this arm, a gate that refused everything would pass 1-3); +5. the SITING: the refusal must not be swallowed by the blanket ``except Exception`` guarding the + rotation-meta reconcile in ``Engine.start``. Proved two ways — an AST check that the call is not + lexically inside that handler's ``try``, and the two engine-level tests above, which would both go + green-by-omission (``start()`` returning normally) if the gate were moved beneath it. + +PHI/secret-safe: identifiers, dates and one-way key-ids only, and every key here is generated for the +test. No message bodies are involved on this path at all. +""" + +from __future__ import annotations + +import ast +import datetime +from pathlib import Path +from typing import Any + +import pytest + +from messagefoundry.config.ai_policy import SecurityEnforcement +from messagefoundry.config.settings import ( + AlertsSettings, + AuthSettings, + SecretRotationSettings, + SecuritySettings, + StoreSettings, + security_loosenings, +) +from messagefoundry.pipeline import secret_rotation as sr +from messagefoundry.pipeline.engine import Engine +from messagefoundry.pipeline.secret_rotation import ( + SecretStamp, + StoreKeyRotationOverdueError, + enforce_store_key_expiry, +) +from messagefoundry.store.crypto import generate_key, make_cipher +from messagefoundry.store.store import MessageStore + +_UTC = datetime.UTC +_REF = datetime.datetime(2026, 6, 15, 12, 0, tzinfo=_UTC) +_REF_TS = _REF.timestamp() +_DEK = "MEFOR_STORE_ENCRYPTION_KEY" +_ENGINE_PY = Path(__file__).resolve().parents[1] / "messagefoundry" / "pipeline" / "engine.py" + + +class _RecordingSink: + """Records ``secret_rotation_due`` calls; every other AlertSink method is inert.""" + + def __init__(self) -> None: + self.calls: list[dict[str, Any]] = [] + + def secret_rotation_due( + self, + name: str, + *, + secret: str, + last_rotated: str, + days_overdue: int, + enforced: bool = False, + ) -> None: + self.calls.append( + { + "name": name, + "secret": secret, + "last_rotated": last_rotated, + "days_overdue": days_overdue, + "enforced": enforced, + } + ) + + def __getattr__(self, _name: str) -> Any: + return lambda *a, **k: None + + +def _stamps(last_rotated: datetime.date) -> dict[str, SecretStamp]: + return { + _DEK: SecretStamp( + secret=_DEK, + label="store data-encryption key", + fingerprint="dekid-aaa", + tracked_since=datetime.date(2025, 1, 1), + last_rotated=last_rotated, + max_age_days=365, + ) + } + + +def _enforce(**over: Any) -> None: + """Run the gate at the fixed reference clock with ENFORCE + a keyed store unless overridden.""" + kw: dict[str, Any] = { + "settings": SecretRotationSettings(), + "stamps": _stamps(datetime.date(2025, 4, 21)), # 420 days before _REF + "enforcement": SecurityEnforcement.ENFORCE, + "dek_key_id": "dekid-aaa", + "alert_sink": None, + "now": _REF_TS, + } + kw.update(over) + enforce_store_key_expiry( + kw.pop("settings"), + kw.pop("stamps"), + **kw, + ) + + +# --- ARM 1: an overdue calendar age REFUSES ---------------------------------------------------- + + +def test_overdue_calendar_age_refuses() -> None: + """420 days old, 365 max + 30 grace = 395 -> 25 days past grace. Before #1004 this only alerted.""" + with pytest.raises(StoreKeyRotationOverdueError) as exc: + _enforce() + assert exc.value.days_overdue == 55 # 420 - 365, the same figure the escalation alert reports + assert exc.value.last_rotated == "2025-04-21" + # The message must name the remedy, not just the fault: an operator meeting this at 3am needs the + # two ways out (rotate, or accept the risk explicitly) in the line that stopped them. + text = str(exc.value) + assert "rotate-key" in text + assert "enforce_store_key_expiry = false" in text + + +def test_the_operator_override_date_drives_the_refusal() -> None: + """`store_key_last_rotated` wins over the stamp — the first-start trip case the item requires.""" + with pytest.raises(StoreKeyRotationOverdueError): + _enforce( + settings=SecretRotationSettings(store_key_last_rotated="2025-01-01"), # 530 days + stamps={}, # no stamp at all: a fresh install whose operator declared a real prior date + ) + + +def test_a_short_max_age_trips_at_first_start() -> None: + """The other first-start trip case: shipped stamp, operator-shortened max age.""" + with pytest.raises(StoreKeyRotationOverdueError): + _enforce( + settings=SecretRotationSettings(store_key_max_age_days=1, enforce_grace_days=0), + stamps=_stamps(datetime.date(2026, 6, 1)), # 14 days old, max 1 + grace 0 + ) + + +# --- ARM 2: an UNDETERMINED age REFUSES -------------------------------------------------------- + + +def test_undetermined_age_refuses_and_alerts() -> None: + """No stamp and no operator date, on a KEYED store that tracks rotation meta: the reconcile failed + and was swallowed upstream. An undetermined age is not a young one — it refuses, and it says why.""" + sink = _RecordingSink() + with pytest.raises(StoreKeyRotationOverdueError) as exc: + _enforce(stamps={}, alert_sink=sink) + assert exc.value.last_rotated == "unknown" + assert exc.value.days_overdue is None + assert "could not be determined" in str(exc.value) + # The alert is required IN ADDITION, never instead — nothing else recorded WHY the engine stopped. + assert len(sink.calls) == 1 + assert sink.calls[0]["enforced"] is True + assert sink.calls[0]["last_rotated"] == "unknown" + + +def test_a_broken_sink_cannot_swallow_the_undetermined_refusal() -> None: + """The escalation alert's own try/except catches a SINK failure. That containment must not reach the + stop: a notifier outage that silently re-enabled an expired key would be the defect one layer down.""" + + class _BrokenSink(_RecordingSink): + def secret_rotation_due(self, *a: Any, **k: Any) -> None: + raise RuntimeError("notifier is down") + + with pytest.raises(StoreKeyRotationOverdueError): + _enforce(stamps={}, alert_sink=_BrokenSink()) + + +# --- ARM 3: the opt-out disables the refusal, and announces itself ----------------------------- + + +def test_opt_out_suppresses_the_refusal_but_keeps_the_alert() -> None: + """`enforce_store_key_expiry = false` drops the RAISE and nothing else. The reminder still fires on + the undetermined branch — an operator who accepted the risk still has to be told the key is stale.""" + sink = _RecordingSink() + _enforce( # must not raise + settings=SecretRotationSettings(enforce_store_key_expiry=False), + stamps={}, + alert_sink=sink, + ) + assert len(sink.calls) == 1, "the opt-out must not also silence the reminder" + + +def test_opt_out_suppresses_the_overdue_refusal() -> None: + _enforce(settings=SecretRotationSettings(enforce_store_key_expiry=False)) # must not raise + + +def test_the_overdue_branch_does_not_double_alert_on_the_normal_path() -> None: + """A populated stamp map means the reconcile completed, so `_maybe_escalate_dek` has ALREADY sent + the enforced alert on this exact condition. A second one here would train operators to ignore it.""" + sink = _RecordingSink() + with pytest.raises(StoreKeyRotationOverdueError): + _enforce(alert_sink=sink) # default stamps: 420 days old, reconcile succeeded + assert sink.calls == [] + + +def test_the_overdue_branch_DOES_alert_when_nothing_else_has() -> None: + """The narrow silent path: the reconcile failed (no stamps) AND the operator set + `store_key_last_rotated`, so there is a date to judge but no escalation alert went out. With the + opt-out ON the raise is loud enough; with it OFF this alert is the only remaining signal.""" + sink = _RecordingSink() + _enforce( + settings=SecretRotationSettings( + store_key_last_rotated="2025-01-01", enforce_store_key_expiry=False + ), + stamps={}, + alert_sink=sink, + ) + assert len(sink.calls) == 1, "an overdue key with the opt-out on must not start in silence" + assert sink.calls[0]["last_rotated"] == "2025-01-01" # a real date, not "unknown" + assert sink.calls[0]["enforced"] is True + + +def test_the_opt_out_is_a_NAMED_security_loosening() -> None: + """A silent opt-out from a refusal is indistinguishable from the refusal never having been built.""" + named = dict( + security_loosenings( + SecuritySettings(), + StoreSettings(), + AuthSettings(), + AlertsSettings(), + SecretRotationSettings(enforce_store_key_expiry=False), + (), + (), + (), + ) + ) + assert "enforce_store_key_expiry" in named + # The entry must say what the SITE gives up, not that a setting is off. + risk = named["enforce_store_key_expiry"] + assert "calendar" in risk.lower() + assert "alert" in risk.lower() + + +def test_the_shipped_default_is_not_reported_as_a_loosening() -> None: + """Non-vacuity for the arm above: at the default the registry must stay silent, or it is noise.""" + named = [ + n + for n, _ in security_loosenings( + SecuritySettings(), + StoreSettings(), + AuthSettings(), + AlertsSettings(), + SecretRotationSettings(), + (), + (), + (), + ) + ] + assert named == [] + assert SecretRotationSettings().enforce_store_key_expiry is True + + +# --- ARM 4: a key that is NOT overdue is unaffected -------------------------------------------- + + +def test_a_young_key_is_untouched() -> None: + """Without this arm, a gate that refused unconditionally would pass every test above.""" + _enforce(stamps=_stamps(datetime.date(2026, 6, 1))) # 14 days old + + +def test_within_the_grace_window_is_untouched() -> None: + """Past max age but inside the enforcement grace: still an alert-only condition, as before.""" + _enforce(stamps=_stamps(datetime.date(2025, 5, 31))) # 380 days: past 365, inside 395 + + +def test_no_refusal_under_warn_enforcement() -> None: + """WARN is the other dial position (there is no OFF). The gates warn and continue there, and this + refusal follows the same dial as the ENFORCE escalation alert it rides beside.""" + _enforce(enforcement=SecurityEnforcement.WARN, stamps=_stamps(datetime.date(2024, 1, 1))) + _enforce(enforcement=SecurityEnforcement.WARN, stamps={}) # undetermined, too + + +def test_a_keyless_store_has_no_dek_to_expire() -> None: + """No local DEK (keyless, or `vault_transit`) means no calendar axis — and an undetermined age + there is the ORDINARY state, not a swallowed failure. Refusing would break every keyless start.""" + _enforce(dek_key_id=None, stamps={}) + + +# --- ARM 5: the SITING, which is what makes this a control ------------------------------------ + + +def _calls_named(node: ast.AST, name: str) -> list[ast.Call]: + return [ + n + for n in ast.walk(node) + if isinstance(n, ast.Call) and isinstance(n.func, ast.Name) and n.func.id == name + ] + + +def _guarded_reconcile_try(tree: ast.Module) -> ast.Try: + """The ``try`` in engine.py whose body awaits ``reconcile_rotation_meta`` — the blanket handler + whose entire body is a log call. Located by CONTENT, so it survives any line-number churn.""" + for node in ast.walk(tree): + if not isinstance(node, ast.Try): + continue + if _calls_named(node, "reconcile_rotation_meta"): + return node + raise AssertionError( + "no try/except around reconcile_rotation_meta in engine.py — this test's premise is gone; " + "re-derive the siting rather than deleting the check" + ) + + +def test_the_reconcile_handler_is_still_the_blanket_one_this_test_guards_against() -> None: + """POSITIVE CONTROL. Everything below is worthless if this handler stopped being a blanket + swallow — the test would then be pinning the siting of a gate against a handler that re-raises.""" + tree = ast.parse(_ENGINE_PY.read_text(encoding="utf-8")) + guarded = _guarded_reconcile_try(tree) + assert guarded.handlers, "the reconcile await is no longer guarded at all" + for handler in guarded.handlers: + assert isinstance(handler.type, ast.Name) and handler.type.id == "Exception", ( + "the reconcile handler narrowed — re-read the siting decision" + ) + assert not any(isinstance(n, ast.Raise) for n in ast.walk(handler)), ( + "the reconcile handler now re-raises; the swallow this test guards against is gone" + ) + + +def test_the_refusal_is_sited_OUTSIDE_the_blanket_reconcile_handler() -> None: + """THE MUTATION CONTROL. Move the ``enforce_store_key_expiry`` call inside the try above and this + goes red. A refusal beneath that handler is logged and stepped over: a traceback, not a control.""" + tree = ast.parse(_ENGINE_PY.read_text(encoding="utf-8")) + guarded = _guarded_reconcile_try(tree) + assert _calls_named(guarded, "enforce_store_key_expiry") == [], ( + "the store-key expiry refusal is sited INSIDE the blanket `except Exception` guarding the " + "rotation-meta reconcile, whose entire body is a log call. It would be swallowed and the " + "engine would start on an expired key. Site it after the handler (BACKLOG #1004, trap 1)." + ) + # ...and it must exist somewhere, or the assertion above passes by absence. + assert _calls_named(tree, "enforce_store_key_expiry"), ( + "engine.py never calls enforce_store_key_expiry — the gate is gone, not merely re-sited" + ) + + +# --- ARM 5 (behavioural): the refusal propagates out of Engine.start() ------------------------- + + +async def _start_engine(tmp_path: Path, settings: SecretRotationSettings) -> None: + """Start a real Engine over a real KEYED SQLite store and let any refusal escape.""" + store = await MessageStore.open(tmp_path / "expiry.db", cipher=make_cipher(generate_key())) + engine = Engine( + store, + secret_rotation_settings=settings, + security_enforcement=SecurityEnforcement.ENFORCE, + ) + try: + await engine.start() + finally: + try: + await engine.stop() + except ( + Exception + ): # a refused start leaves a partially wired engine; teardown is best-effort + await store.close() + + +async def test_an_overdue_key_aborts_Engine_start(tmp_path: Path) -> None: + """END TO END, and this is also a siting proof: sited inside the blanket handler, `start()` would + return normally and `pytest.raises` would fail. A green here is evidence only because of that.""" + with pytest.raises(StoreKeyRotationOverdueError): + await _start_engine( + tmp_path, + SecretRotationSettings(store_key_last_rotated="2020-01-01"), # far past 365 + 30 + ) + + +async def test_an_undetermined_age_aborts_Engine_start( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + """TRAP 1b, the second-order swallow. Force the reconcile to raise: the blanket handler eats it, + the stamps stay empty, and a gate written to read them would silently not fire. It must refuse.""" + + async def _boom(*_a: Any, **_k: Any) -> dict[str, SecretStamp]: + raise RuntimeError("meta store is unreachable") + + monkeypatch.setattr("messagefoundry.pipeline.engine.reconcile_rotation_meta", _boom) + with pytest.raises(StoreKeyRotationOverdueError): + await _start_engine(tmp_path, SecretRotationSettings()) + + +async def test_the_opt_out_lets_an_overdue_engine_start(tmp_path: Path) -> None: + """The escape has to actually work, or it is not an opt-out.""" + await _start_engine( + tmp_path, + SecretRotationSettings(store_key_last_rotated="2020-01-01", enforce_store_key_expiry=False), + ) + + +async def test_the_opt_out_lets_an_undetermined_engine_start( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch +) -> None: + async def _boom(*_a: Any, **_k: Any) -> dict[str, SecretStamp]: + raise RuntimeError("meta store is unreachable") + + monkeypatch.setattr("messagefoundry.pipeline.engine.reconcile_rotation_meta", _boom) + await _start_engine(tmp_path, SecretRotationSettings(enforce_store_key_expiry=False)) + + +async def test_a_fresh_keyed_engine_starts_on_the_shipped_defaults(tmp_path: Path) -> None: + """ARM 4 at the engine level. The shipped configuration cannot be surprised: a first keyed start + stamps the DEK as new (tracked_since is an age FLOOR), so nothing can trip for 395 days.""" + await _start_engine(tmp_path, SecretRotationSettings()) + + +def test_the_gate_and_the_alert_read_the_SAME_overdue_expression() -> None: + """One arithmetic, not two. If the refusal and the ENFORCE alert could disagree about "overdue", + an operator would meet a stop with no matching alert — or an alert with no stop.""" + sink = _RecordingSink() + stamps = _stamps(datetime.date(2025, 4, 21)) + sr._maybe_escalate_dek( + SecretRotationSettings(), + stamps, + _REF.date(), + enforcement=SecurityEnforcement.ENFORCE, + alert_sink=sink, # type: ignore[arg-type] + ) + with pytest.raises(StoreKeyRotationOverdueError) as exc: + _enforce(stamps=stamps) + assert sink.calls[0]["days_overdue"] == exc.value.days_overdue