feat(security)!: retire handles_real_patient_data; every instance carries patient data (BACKLOG #1279)

.github/workflows/ci.yml

@@ -2587,33 +2587,35 @@ jobs:
           MEFOR_STORE_PASSWORD: "Str0ng_P@ssw0rd!"
           MEFOR_STORE_TRUST_SERVER_CERTIFICATE: "true"
           MEFOR_ALLOW_INSECURE_TLS: "1"  # trusted-network escape for the container's self-signed cert
-          # DECLARE THE DATA CLASS HONESTLY (ADR 0148 GIVEN 1 + docs/SECURITY-LOOSENING.md
-          # "`handles_real_patient_data = false`"). This leg processes harness-GENERATED synthetic HL7 on
-          # a throwaway service container; asserting PHI was a false declaration about the instance, and
-          # the register names exactly this case as acceptable: "a CI runner ... that only ever processes
-          # synthetic / sample HL7". Since GIVEN 1 the built-in `dev` env derives PHI, so a genuinely
-          # throwaway CI box must now set this EXPLICITLY — it is no longer the `dev` default.
+          # THE DATA-CLASS DECLARATION THIS LEG USED TO CARRY IS GONE (BACKLOG #1279). Every instance
+          # carries patient data now, so `handles_real_patient_data = false` no longer exists and this
+          # leg names the ONE gate it actually needs relaxed instead.
           #
-          # It is also what makes the leg run at all. This job was RED for four consecutive nights
-          # (2026-07-27..30) on:
+          # THE GATE, and why it is the enforcement dial and not something narrower. This job was RED
+          # for four consecutive nights (2026-07-27..30) on:
           #     ValueError: SQL Server TLS is weakened (trust_server_certificate=true or encrypt=false)
-          # Under enforcing-PHI the `MEFOR_ALLOW_INSECURE_TLS` escape above is clamped INERT by design
-          # (#200 / ADR 0092 decision 2 — `weakened_tls_escape_permitted`), so a self-signed container
-          # can never be trusted from a PHI instance. The previous comment here said this leg "validates
-          # PHI+enforce green, not a synthetic opt-out" — an intent that is UNREACHABLE with a
-          # `services:` container: GitHub starts it before any step runs, so a cert generated in a step
-          # cannot be mounted into it. Whoever set that provisioned the PHI posture's retention and
-          # egress requirements but not its TLS one.
+          # `weakened_tls_escape_permitted` clamps `MEFOR_ALLOW_INSECURE_TLS` INERT while the instance
+          # is enforcing (#200 / ADR 0092 decision 2), so the escape above does nothing under the
+          # shipped `enforce`. The clamp used to require enforcing AND PHI, and declaring the box
+          # synthetic was how this leg escaped it; with every instance PHI the only key left is the
+          # enforcement dial. `warn` honors the escape and keeps every gate reporting.
           #
-          # WHAT THIS GIVES UP, stated plainly: the leg no longer exercises the PHI+enforce path. Its
+          # It is what makes the leg run at all, and the reason a narrower fix does not exist here: the
+          # container's certificate is self-signed and GitHub starts a `services:` container before any
+          # step runs, so a cert generated in a step cannot be mounted into it. There is no per-hop
+          # attestation for the store hop to carry instead.
+          #
+          # WHAT THIS GIVES UP, stated plainly: the leg no longer exercises the enforcing path. Its
           # actual job is a load/throughput smoke of the SQL Server store, and `sqlserver-store` carries
-          # the functional coverage. Restoring PHI+enforce here needs a REAL certificate — the job moved
+          # the functional coverage. Restoring `enforce` here needs a REAL certificate — the job moved
           # off `services:` onto a `docker run` with a generated cert mounted and trusted — which is
-          # worth doing deliberately, not as a side effect of unbreaking a nightly.
+          # worth doing deliberately, not as a side effect of unbreaking a nightly. Note this is now a
+          # NARROWER relaxation than the one it replaces: the retired declaration silenced nineteen
+          # gates, `enforcement: warn` downgrades them to warnings and silences none.
           #
-          # The PHI-shaped provisions below are KEPT even though synthetic does not require them: they
-          # cost nothing and keep the leg measuring a realistic configuration.
-          MEFOR_SECURITY_HANDLES_REAL_PATIENT_DATA: "false"
+          # The provisions below are now REQUIRED rather than courtesy: under the PHI posture a keyless
+          # start, an unbounded retention window and an unlisted egress each have their own gate.
+          MEFOR_SECURITY_ENFORCEMENT: "warn"
           # Bounded PHI-body retention windows + a locked-down egress allowlisting the loopback sink.
           # The store key is minted at runtime below. The security-notification gate is skipped because
           # auth is off (no accounts to notify).

CHANGELOG.md

@@ -36,6 +36,31 @@ All notable changes to MessageFoundry are documented here. The format follows
   ([`DEPLOY-SERVER-DB.md`](docs/DEPLOY-SERVER-DB.md) §1.2), which it previously was not.
   ([BACKLOG #1008](docs/BACKLOG.md))
 
+### Removed
+- **BREAKING: `[security].handles_real_patient_data` is gone, and with it the whole data-class axis.**
+  Every instance carries patient data; the PHI gates apply unconditionally. Setting the key — or its
+  pre-ADR-0118 spelling `[ai].data_class` — now **refuses at load** with a message naming the switch to
+  reach for instead. Removed with it: the `DataClass` enum, `HopPosture.is_phi`, the `data_class` and
+  `synthetic_relaxation` fields on `SecurityPosture`, and `data_class` on `AiPolicy`.
+  `derived_posture()` / `require_posture()` return the production tier alone.
+  **Why, in one line: it turned off nineteen start-up gates on one line, and it was not the audited
+  opt-out the documentation claimed.** `security_loosenings()` never named it, so the serve-time
+  loosening warning — the thing that fires for every other deviation — did not fire for the widest
+  relaxation the product shipped. The completeness test that should have caught that exempted the field
+  with a reason that was false, in an exemption branch that could never execute.
+  **What to use instead:** the gate you actually mean. Each is separately named, separately audited and
+  separately reported — `allow_unencrypted_phi` (plus `allow_unencrypted_phi_under_strict_enforcement`
+  under the shipped `enforcement = enforce`), `block_unlisted_outbound`,
+  `allow_keeping_phi_indefinitely`, `allow_single_factor_admin_when_exposed`,
+  `allow_unverified_alert_smtp_tls`, `[alerts].security_notifications_required`, a per-connection
+  `cleartext_accepted` / `tls_revocation_attested`, or the `[security].enforcement` dial.
+  **What this costs:** a box that ran key-free on the declaration now needs a key or the audited
+  per-gate ack. Nothing is deployed (there is no migration), and both in-repo users of the declaration —
+  CI's SQL Server load leg and the failover load harness — moved to per-gate relaxations that are
+  *narrower* than what they replace. See
+  [ADR 0186](docs/adr/0186-retire-the-synthetic-data-declaration-every-instance-carries-patient-data.md)
+  and BACKLOG #1279.
+
 ### Changed
 - **Web console engine UI seam `93ba1f10b9dccfc8` -> `b93f38d097f97a45`.** `SecurityPosture` gained the
   additive `store_privilege` object above, and `StorePrivilegeView` joins the discovered surface.

docs/BACKLOG.md

@@ -17042,9 +17042,10 @@ measurement from this row's subject and it is named here rather than performed.*
 >
 > **Two failures appeared in BOTH arms and are pre-existing, not sandbox-related.**
 > `OB_IMMUNIZATION_BODYCRED` and `OB_IMMUNIZATION_REGISTRY` fail to start because `environments/dev.toml`
-> carries none of the `registry_*` values; the engine isolates them and continues. The smoke also needs
-> `[security].handles_real_patient_data = false`, or `serve --env dev` refuses to start without a store
-> encryption key.
+> carries none of the `registry_*` values; the engine isolates them and continues. The smoke also needs a
+> store encryption key, or `serve --env dev` refuses to start. **CORRECTED 2026-09-09 (BACKLOG #1279):**
+> this line said to set `[security].handles_real_patient_data = false`, which the loader now REFUSES.
+> Mint a key, or set `[security].allow_unencrypted_phi` (plus its strict-enforcement ack).
 >
 > **THE DOCUMENTATION HALF OF THIS ROW SHIPPED AND STANDS AT THE CURRENT DEFAULT.** The five findings
 > this row named are now written down, phrased for an opt-in mode rather than a default one. Finding 1
@@ -17153,9 +17154,10 @@ anything about this change.**
 So **"the samples still load and run" is now earned** for the six MLLP sample feeds and the X12 one.
 Two pre-existing failures appear in every run including the `mode=off` control and are **not**
 sandbox-related: `OB_IMMUNIZATION_BODYCRED` and `OB_IMMUNIZATION_REGISTRY` fail to build because
-`environments/dev.toml` carries none of the `registry_*` values. Note the smoke needs
-`[security].handles_real_patient_data = false`, or `serve --env dev` refuses to start without a store
-encryption key.
+`environments/dev.toml` carries none of the `registry_*` values. Note the smoke needs a store encryption
+key, or `serve --env dev` refuses to start. **CORRECTED 2026-09-09 (BACKLOG #1279):** this line said to
+set `[security].handles_real_patient_data = false`, which the loader now REFUSES; mint a key, or set
+`[security].allow_unencrypted_phi` (plus its strict-enforcement ack).
 
 *FOUR CONSEQUENCES THIS ROW DOES NOT NAME, EACH FOUND BY READING THE SHIPPED CODE.* Any of them can
 turn "one default plus a release note" into something a reader would have been misled by.
@@ -17226,7 +17228,23 @@ note the row did not budget for. Suggest **difficulty 5-6**, and dispatch it to
 can finish a suite rather than merely a lane with hours.
 ## 1279. treat every instance as carrying patient data and retire the synthetic-data declaration
 
-> 🔢 **Re-scored 2026-08-20 -> P2.** Value **6/10** · Difficulty **6/10** · _big bet_. The opt-out still ships at settings.py:3738 and still translates to the enum at :4234-4235, while the comment at :3733-3735 continues to contradict :2318, which has said PHI since ADR 0148. Value is a secure-defaults simplification rather than a shipped-default defect, since only an explicit declaration loses the refusals; difficulty stays high on surface alone -- 77 data_class occurrences across the engine plus 45 in tests, and the api/models.py wire-contract change -- with no migration cost added per section 0. _(previously unscored.)_
+> ✅ **SHIPPED 2026-09-09 -- owner ruling, given directly: "I don't want to use handles_real_patient_data any more. I want mefor to always take a PHI posture as the default. Users can adjust individual settings as they need, but not use handles_real_patient_data = false as a combined override." Recorded as [ADR 0186](adr/0186-retire-the-synthetic-data-declaration-every-instance-carries-patient-data.md).**
+>
+> **NOT YET ON `main` when this line was written.** It lands with the PR that carries this edit; a reader who needs it verified should check `main` rather than trust this banner, which is the builder's claim about its own branch.
+>
+> **WHAT LANDED.** `[security].handles_real_patient_data`, `[ai].data_class` and the `DataClass` enum are removed. `HopPosture` loses `is_phi`; `derived_posture()`/`require_posture()` return the production tier alone; `SecurityPosture` drops `data_class` + `synthetic_relaxation` and `AiPolicy` drops `data_class`. Both key spellings are REFUSED at load (`_REMOVED_KEYS`) with a message naming the per-gate switches, rather than ignored -- a config asserting the gates are off while the engine runs them all is a silent contradiction. CI's SQL Server load leg and the failover harness take per-gate relaxations; both are NARROWER than the declaration they replace.
+>
+> **THREE THINGS THIS ROW DID NOT KNOW, ALL MEASURED AT `0ce6d95cf` BEFORE THE CHANGE.**
+>
+> 1. **The lever was not the audited opt-out the docs claimed, and the test that should have caught that could not fire.** `docs/SECURITY-LOOSENING.md` said it was *"named by `security_loosenings()`, surfaced in `GET /security/posture`, and warned at `serve`"*. `security_loosenings()` spans 364 lines and contained ZERO occurrences of `handles_real_patient_data`, `data_class`, `DataClass` or `synthetic`. The serve-time loosening warning reads that registry, so it never fired for the widest relaxation shipped. `tests/test_security_posture_defaults.py` exempted the field with the reason *"the data-class lever has its own entry keyed on the derived posture"* -- there was no such entry, and the exemption was also UNREACHABLE, because the completeness loop skips any field whose default is not a `bool` and this one defaults `None`. Two lines of dead code carrying a false statement about a security control, inside the test that exists to prevent exactly that (SDS-3.7).
+> 2. **The row's count of nineteen is right, and it was re-derived independently rather than trusted.** Sixteen branches in `_serve`, one lifespan check in `api/app.py`, two dispositions in `config/tls_policy.py`. Eight are hard refusals under the shipped `enforcement = enforce`.
+> 3. **The row's own line anchors had drifted** (`settings.py:3730`/`:3738` for one field across two scorings). Every site here was re-located by symbol. The row's substantive claims all held.
+>
+> **THE STALE COMMENT THE ROW FLAGGED WAS STILL THERE AND IS FIXED** -- `settings.py` described the derivation as `dev` -> synthetic, contradicting `_KNOWN_ENV_POSTURE` since ADR 0148. Two further stale doc lines went with it: `SECURITY-LOOSENING.md` claimed declaring a box synthetic was the only way to silence a PHI cleartext hop (ADR 0153 had removed that arm), and `DEPLOYMENT.md` claimed the declaration was audited in `security_loosenings()`.
+>
+> **ONE RESIDUAL, FILED AND UNALLOCATED, and it is a real gap rather than tidy-up.** ADR 0153 left `api_phi_hop_disposition` and `forward_hop_disposition` keyed on the data label because neither cell is a connection and so neither can carry a per-hop `cleartext_accepted`. Removing the label resolves those carve-outs by subtraction and makes 0153's recorded follow-up load-bearing: under `enforce`, an unproven API serve hop and an unattested plaintext log-forwarding hop now have NO per-cell way to accept a risk. The `[security]`-level declaration for each is unbuilt. Name the subject, not a number -- it is unallocated.
+>
+> _Original filing follows._ **Re-scored 2026-08-20 -> P2.** Value **6/10** · Difficulty **6/10** · _big bet_. The opt-out still ships at settings.py:3738 and still translates to the enum at :4234-4235, while the comment at :3733-3735 continues to contradict :2318, which has said PHI since ADR 0148. Value is a secure-defaults simplification rather than a shipped-default defect, since only an explicit declaration loses the refusals; difficulty stays high on surface alone -- 77 data_class occurrences across the engine plus 45 in tests, and the api/models.py wire-contract change -- with no migration cost added per section 0. _(previously unscored.)_
 >
 > **Filed 2026-08-16 - not started. THE PRODUCT LETS AN OPERATOR DECLARE THAT AN INSTANCE HOLDS ONLY SYNTHETIC DATA, AND THAT DECLARATION UNLOCKS NINETEEN START-UP RELAXATIONS.** The lever is `[security].handles_real_patient_data` ([`config/settings.py:3730`](../messagefoundry/config/settings.py)), translated to `[ai].data_class` at `settings.py:4226-4227`, over the `DataClass` enum at [`config/ai_policy.py:66-75`](../messagefoundry/config/ai_policy.py). **THE CHANGE: remove the distinction entirely and treat every instance as carrying patient data.**
 > **THIS REMOVES AN OPT-OUT; IT DOES NOT FLIP A DEFAULT -- state it that way or the item overstates itself.** `dev` **already** derives the patient-data posture: `_KNOWN_ENV_POSTURE["dev"] = (DataClass.PHI, False)` (`settings.py:2310`, [ADR 0148](adr/0148-phi-default-posture-and-an-explicit-security-enforcement-level.md) GIVEN 1), and `serve` requires an environment (`settings.py:2348-2350`, enforced by `require_posture()` at `__main__.py:1191`). **A stock development box is treated as carrying patient data today.** The only configurations that are not are those that explicitly declared `handles_real_patient_data = false`. Those are what stop working.