feat(tls): let the HTTP egress family and both Vault hops name an internal CA (BACKLOG #1180)

docs/ASVS-L2-PHASE0-CHANGES.md

@@ -446,8 +446,8 @@ tables in [`CONNECTIONS.md`](CONNECTIONS.md) §"Resource management & limits" (A
 | SMART Backend Services token endpoint (ADR 0024) | outbound | HTTPS POST to the operator-pinned token URL (no discovery); routed through the connection's forward proxy when one resolves for the **token** host | the REST verifying no-redirect opener; a cleartext-`http` token URL is **refused** unless `MEFOR_ALLOW_INSECURE_TLS` (the client assertion is a credential) | a signed `client_assertion` JWT (`smart_private_key`, optionally passphrase-protected) | **yes** — `smart_token_url` per Connection | `smart_token_url`, `smart_client_id`, `smart_private_key`, `smart_scope`, `smart_timeout_seconds`, `smart_expiry_skew_seconds` |
 | Generic OAuth2 client-credentials token endpoint | outbound | HTTPS POST for the non-SMART REST/SOAP/FHIR/DICOMweb bearer path; also proxy-routed per token host | the same verifying no-redirect opener; a cleartext-`http` credential hop is posture-keyed and refused | `oauth2_client_id` + `oauth2_client_secret`, sent `basic` or `post` per `oauth2_auth_style` | **yes** — `oauth2_token_url` per Connection | `oauth2_token_url`, `oauth2_client_id`, `oauth2_client_secret`, `oauth2_scope`, `oauth2_auth_style`, `oauth2_timeout_seconds` |
 | Engine-brokered AI assistance (ADR 0135) | outbound | HTTPS POST of a `code_only` assist prompt to a **customer-managed / self-hosted** LLM endpoint; runs off the event loop | the REST verifying no-redirect opener; a cleartext-`http` endpoint carrying the key is refused unless `MEFOR_ALLOW_INSECURE_TLS` | `MEFOR_AI_API_KEY`, sent as the `x-api-key` header | **yes** — verbatim the requirement's "the end user provides an external location" | `[ai].endpoint`, `[ai].api_key`, `[ai].allowed_endpoints` (a **dedicated fail-closed** SSRF allowlist — an EMPTY list refuses everything; deliberately **not** `[egress].allowed_http`), `[ai].provider`, `[ai].model` |
-| HashiCorp Vault Transit — store DEK envelope-decrypt (ADR 0019) | outbound | HTTPS via `hvac` (the `[vault]` extra); port from the address | TLS verification is `hvac`/`requests`' own default — the engine sets no explicit client TLS options here | a Vault token — `MEFOR_STORE_VAULT_TOKEN` (`hvac` falls back to `VAULT_TOKEN` when unset) | **yes** — `MEFOR_STORE_VAULT_ADDR` (opt-in; fail-closed) | `MEFOR_STORE_VAULT_ADDR`, `MEFOR_STORE_VAULT_TOKEN`, `MEFOR_STORE_VAULT_TRANSIT_KEY` |
-| HashiCorp Vault Transit — **bulk at-rest cipher** (`[store].cipher_provider = vault_transit`, ADR 0138) | outbound (**per store operation**) | HTTPS via the same shared `hvac` client build; port from the address. **One `encrypt_data` / `decrypt_data` round trip per encrypted CELL** on every store write and read, plus one `generate_hmac` per audit row — not a startup-only hop | as the DEK hop: `hvac`/`requests` defaults, no engine-set client TLS options | the same Vault token (`MEFOR_STORE_VAULT_TOKEN`) | **yes** — `MEFOR_STORE_VAULT_ADDR` (shared with the DEK hop) | `[store].cipher_provider`, `MEFOR_STORE_TRANSIT_KEY`, `MEFOR_STORE_TRANSIT_AUDIT_KEY` |
+| HashiCorp Vault Transit — store DEK envelope-decrypt (ADR 0019) | outbound | HTTPS via `hvac` (the `[vault]` extra); port from the address | TLS verification defaults to `hvac`/`requests`' own **public** bundle. `MEFOR_STORE_VAULT_CA_FILE` (BACKLOG #1180, ASVS 12.3.4) narrows it to one internal CA, resolved through the same `resolve_trust_anchor` every other outbound hop uses; unset, the client is constructed exactly as before | a Vault token — `MEFOR_STORE_VAULT_TOKEN` (`hvac` falls back to `VAULT_TOKEN` when unset) | **yes** — `MEFOR_STORE_VAULT_ADDR` (opt-in; fail-closed) | `MEFOR_STORE_VAULT_ADDR`, `MEFOR_STORE_VAULT_TOKEN`, `MEFOR_STORE_VAULT_TRANSIT_KEY`, `MEFOR_STORE_VAULT_CA_FILE` |
+| HashiCorp Vault Transit — **bulk at-rest cipher** (`[store].cipher_provider = vault_transit`, ADR 0138) | outbound (**per store operation**) | HTTPS via the same shared `hvac` client build; port from the address. **One `encrypt_data` / `decrypt_data` round trip per encrypted CELL** on every store write and read, plus one `generate_hmac` per audit row — not a startup-only hop | as the DEK hop — same shared client build, so `MEFOR_STORE_VAULT_CA_FILE` narrows this hop too | the same Vault token (`MEFOR_STORE_VAULT_TOKEN`) | **yes** — `MEFOR_STORE_VAULT_ADDR` (shared with the DEK hop) | `[store].cipher_provider`, `MEFOR_STORE_TRANSIT_KEY`, `MEFOR_STORE_TRANSIT_AUDIT_KEY` |
 | DR backup destination (ADR 0049) | outbound (scheduled + on-demand) | local filesystem, or **SMB/CIFS over TCP when `[backup].destination` is a UNC path** (the OS redirector owns the port); a cloud URL is **rejected at load** | n/a — no engine-terminated TLS on this hop; SMB dialect security is the OS's | the engine service account's **own** identity — `[backup]` exposes no `credential_*` impersonation knob, unlike the FILE connector | **yes** — `[backup].destination` | `[backup].enabled`, `[backup].destination`, `schedule_at`, `retention_keep`, `snapshot_method`, `allow_unencrypted` |
 | Security-event notification email, per user | outbound | SMTP through the **same** `[alerts]` transport and default port 587, but a **second, independent** background dispatcher — its own 1000-item queue and its own drain task — mailing each affected USER's own address, not the operator `email_to` list | STARTTLS **and certificate verification**, as the operator sink — plumbed at this call site in its own right (`pipeline/security_notify.py`), not inherited implicitly | as the operator sink | **yes** — the same `[alerts].email_smtp_host` | `[auth].notify_security_events`, `[alerts].email_*` |
 | HashiCorp Vault KV v2 — connector-credential secrets provider (ADR 0019) | outbound | HTTPS via `hvac`, a **separate client** from the Transit one behind the same extra | as above | a Vault token — `MEFOR_SECRETS_VAULT_TOKEN` (falls back to `VAULT_TOKEN`) | **yes** — `MEFOR_SECRETS_VAULT_ADDR` (opt-in; fail-closed) | `MEFOR_SECRETS_VAULT_ADDR`, `MEFOR_SECRETS_VAULT_TOKEN`, `[secrets].provider` |

docs/BACKLOG.md

@@ -10637,6 +10637,16 @@ Nothing here touches the TLS/FTPS context in the same module, which was already
 
 **Still not an honest pass:** the item's two traps hold -- shipping `trust_anchor_mode = "pinned"` as the default is refused at config load without an internal CA (`config/settings.py:1025-1031`) so every stock configuration would fail to start, and arguing that a hospital's OS trust store IS the enterprise CA is a property of the operator's image build laundered into a product control. The sharpest trap is a third the item does not name: the cell's own re-score trigger says to watch for the anchor resolver or a CA parameter appearing on those HTTP factories, so **the expressibility work trips that trigger by construction, and if it lands and the cell is re-scored while every default is still permissive, that is wiring an existing toolkit into the pipeline so an absence claim stops firing -- wearing the hat of a trigger the cell wrote for itself.** Expressibility must never be scored without the forcing control. Two more, both reachable from the record: re-scoring on the discovery that the generic ODBC dialect passes operator keywords through (a newly FOUND opt-in capability is not a newly BUILT control, and that dialect is not the default), and re-scoring on the syslog hop (one hop of roughly a dozen, reached only by opting into a non-default protocol). Proposed work, unallocated and by subject: the HTTP-egress trust-anchor threading across ALL FOUR factories that lack a certificate parameter, including the lookup executor's own opener map and the separate webhook opener; the Vault client CA argument and scheme gate, with the honest note that it needs new settings and a posture handle on a lazy environment-fed path rather than reusing the shipped ladder; the FTPS factory's lost CA and verification parameters; the DATABASE default-dialect server-certificate parity fix, which the store code already emits against the identical driver; the tray probe CA pin; the IDE certificate-authority setting AND a separate IDE-side forcing rule with its own publicly-certified-peer arm, since no engine-side gate can reach a separate process; the anchor-discipline posture gate, owner-gated and specified against a public-chain-probe predicate wherever measurement is possible with the declaration arm confined to the ODBC and Schannel hops; posture reporting of an unnarrowed internal hop as a first-class loosening; an explicit disposition for the augmenting trust mode, which keeps the OS roots and so does not satisfy the word ONLY; an ECH-sidecar egress ruling rather than leaving it in the unknowns while a pass is claimed; and extending the anchor-integrity preflight beyond the three auth-path anchors.
 
+**Progress 2026-09-04 (builder). THE EXPRESSIBILITY LIMB IS SHIPPED. The default flip and the posture ruling stay open, and this item stays open with them.** Re-measured by execution at HEAD `a2eef0f37`, with a positive control in the same run so an all-negative probe could not pass for a clean repo: `resolve_trust_anchor` was absent from `transports/rest.py`, `soap.py`, `fhir.py` and `dicomweb.py` on all four probes (imported symbol, source mention, context builder, policy plumbing) while `mllp.py`, `dicom.py` and `remotefile.py` returned true on all four; and a recording fake `hvac` showed both `_build_client` sites passing exactly `{url, token, allow_redirects}` and no CA-bearing keyword, with the recorder demonstrably working because it saw those three. Both stated shortfalls reproduce.
+
+**What landed.** One construction point, `build_anchored_https_handler` in `config/tls_policy.py`, now builds every HTTP-family https handler, so the call sites cannot drift. A hop with no CA gets `build_asserted_https_handler` verbatim -- urllib's own context, asserted in place. `augment` loads the internal CA into that same context, so nothing is rebuilt and urllib's ALPN and post-handshake auth are untouched. Only `pinned` (and a per-connection CA, the same shape) substitutes a context, because it must trust ONLY the internal CA and an `SSLContext` cannot unload roots urllib has already loaded; that one arm replays urllib's two measured deltas so pinning a hop does not silently change its handshake. It is reached by `RestDestination`, `SoapDestination` (including the mutual-TLS opener, which builds its own context), `FhirDestination`, `DicomWebDestination`, and the `fhir_lookup` executor's per-connection opener map -- the last has no `Destination` to carry the policy, so `RegistryRunner` threads it in explicitly at both the live and the `build_check` site. Both hvac hops take `MEFOR_STORE_VAULT_CA_FILE` / `MEFOR_SECRETS_VAULT_CA_FILE` through the same resolver, mapped onto `requests`' single `verify=` bundle path; `augment` is REFUSED there rather than silently narrowed, because one path cannot say "public roots plus this CA". **The default is unchanged and it is asserted, not asserted-about:** a stock https destination is still handed the shared `_NO_REDIRECT_OPENER` **by object identity**, and a hop takes a per-connection opener only when it carries an extra handler or an anchor that narrows. Twelve mutations were run against the new assertions and all twelve go red, including two negative controls -- one making an unanchored hop substitute a context, one giving every hop a per-connection opener.
+
+**Two of this item's own claims are corrected by the work.** The severity clause says the Vault hop "would be verified against the full OS store"; the 2026-08-20 research already corrected that to a fail-closed public-bundle hop, and the shipped code confirms it -- `requests` defaults to certifi, not the OS store, so what was missing there was the ability to reach an internal-CA Vault **at all**. And the per-connection `tls_ca_file` this work reads is still unreachable from either authoring surface on these five factories, exactly as this item's own factory probe found for `Ftp()`: precedence rule 1 of `resolve_trust_anchor` is live code on an unspeakable key until those signatures gain the parameter.
+
+**Measured and deliberately NOT done, so nobody reads this as the limb closed.** (1) The `tls_ca_file` parameter on `Rest()`, `Soap()`, `FHIR()`, `DICOMweb()` and `FhirLookup()` -- the connection-scoped half of the same shortfall; the instance `[tls]` anchor is what this pass made expressible. (2) **The SMART token-endpoint hop**, `transports/smart.py`, built from inside the very `FhirDestination.__init__` this pass changed: in `pinned` mode a FHIR destination now anchors its data hop while the hop carrying its `client_assertion` still verifies against the OS store. It is not a one-line thread-through -- the anchor must be resolved for the TOKEN host, which has its own loopback and proxy decisions (ADR 0126 already splits it), so it needs the policy passed into `token_provider_from_settings`, not the destination's resolved anchor. (3) The alert webhook opener in `pipeline/alert_sinks.py`, whose `notifier_from_settings` already receives a `trust_anchor_policy` and threads it to `EmailTransport` only. (4) `[tls].internal_ca_file` reaching the two Vault hops: neither provider has that section in scope -- they hold a `StoreSettings` / `SecretsSettings` and build from the environment -- so it needs threading through `resolve_key_provider` / `resolve_secret_provider`, which is the "new settings" this item's research already named. (5) A per-anchor context cache: with an internal CA set, each anchored lane builds its own `SSLContext` at construction, and `build_check` builds a second one it discards. Startup-only, and caching a mutable `SSLContext` across connectors is its own hazard, so it was left alone rather than done quietly.
+
+**Neither trap was touched and the third one holds.** `trust_anchor_mode` still defaults to `"system"`; nothing was flipped, and the pinned-without-a-CA validator is untouched. The refuse-versus-fall-back posture decision is an owner ruling and is not built. **And this item's own sharpest warning applies to this commit by construction:** the cell's re-score trigger watches for the anchor resolver or a CA parameter appearing on these HTTP factories, and this work fires that trigger deliberately. **Expressibility is not a pass. Do not re-score 12.3.4 on this alone** -- every default is still permissive, and scoring it here would be wiring an existing toolkit into the pipeline so an absence claim stops firing, which is exactly what the record says not to do.
+
 ## 1181. research an honest pass for ASVS 12.3.5 -- intra-service endpoint authentication on an engine whose coordination is store-mediated
 
 > 🔢 **Re-scored 2026-08-20 -> P2.** Value **6/10** · Difficulty **7/10** · _big bet_. The setting is still attestation-only: its sole consumer at settings.py:850 collapses four values to a boolean, so a declared "mtls" changes nothing. Value 6 rather than 7 because the hop the item's own severity names is genuinely closable today -- setting [api].tls_client_ca_file makes the API listener CERT_REQUIRED-verify the proxy's client cert (api/tls.py:58-62) -- so the gap is an unverified declaration beside a working control rather than an unavailable one; difficulty 7 because an honest pass must decide whether an attestation-only setting should exist at all, whether mutual PKI is reachable for the store and Vault hops given the drivers in use, and whether the IDE, tray and apiclient fall inside the requirement's scope. _(was 7/10 · 7/10.)_

docs/CONFIGURATION.md

@@ -1175,7 +1175,11 @@ A provider is consulted **only** for a credential whose per-credential `*_secret
 store password is seam-only (managed identity is preferred there). A reference is `"<kv-path>"` or
 `"<kv-path>#<field>"` for `vault` (field defaults to `value`; KV mount from `MEFOR_SECRETS_VAULT_KV_MOUNT`,
 default `secret`); Vault address/token come from `MEFOR_SECRETS_VAULT_ADDR` / `MEFOR_SECRETS_VAULT_TOKEN`
-(falling back to hvac's `VAULT_ADDR` / `VAULT_TOKEN`). **Fail-closed:** a reference with `provider = none`,
+(falling back to hvac's `VAULT_ADDR` / `VAULT_TOKEN`). Point `MEFOR_SECRETS_VAULT_CA_FILE` at the PEM of
+the CA that issued your Vault server's certificate to verify that hop against your own PKI instead of the
+public bundle `requests` ships with (BACKLOG #1180; the store KeyProvider's twin is
+`MEFOR_STORE_VAULT_CA_FILE`) — a path, not a secret, and unset leaves the hop exactly as it was.
+**Fail-closed:** a reference with `provider = none`,
 an unknown provider, a missing `[vault]` extra, or an unresolvable/empty secret raises at load/connect —
 never a blank credential; the value is never logged.
 

messagefoundry/config/secretprovider_vault.py

@@ -32,6 +32,7 @@
 from typing import TYPE_CHECKING, Any
 
 from messagefoundry.config.secretprovider import SecretProviderError
+from messagefoundry.config.tls_policy import vault_client_verify_kwargs
 
 if TYPE_CHECKING:
     from messagefoundry.config.settings import SecretsSettings
@@ -47,6 +48,13 @@
 _ENV_TOKEN = "MEFOR_SECRETS_VAULT_TOKEN"  # nosec B105 — the env-var NAME, not a token value
 #: KV v2 mount point the connector secrets live under (Vault's conventional default is ``secret``).
 _ENV_KV_MOUNT = "MEFOR_SECRETS_VAULT_KV_MOUNT"
+#: PEM path to the CA that issued the Vault server's certificate (#1180, ASVS 12.3.4). A PATH, not a
+#: secret. The twin of the store KeyProvider's ``MEFOR_STORE_VAULT_CA_FILE``, kept separate for the
+#: same reason the address and token are: the two providers may point at different Vaults. Unset =
+#: hvac's own default, which is ``requests``' PUBLIC certifi bundle.
+#: (:func:`~messagefoundry.config.tls_policy.vault_client_verify_kwargs` records why ``[tls]`` does
+#: not reach this hop yet.)
+_ENV_CA_FILE = "MEFOR_SECRETS_VAULT_CA_FILE"
 
 #: Default field read from a KV secret when a reference omits ``#<field>``.
 _DEFAULT_FIELD = "value"
@@ -67,6 +75,26 @@ def _import_hvac() -> Any:
     return hvac
 
 
+def _vault_ca_kwargs(addr: str | None) -> dict[str, str]:
+    """This Vault hop's ``verify=`` keyword arguments — ``{}`` when no anchor is configured.
+
+    #1180 (ASVS 12.3.4) — the twin of ``store/keyprovider_vault.py``'s, sharing
+    :func:`~messagefoundry.config.tls_policy.vault_client_verify_kwargs`. This client reads connector
+    credentials out of Vault KV, so the anchor that verifies the server is the only thing standing
+    between a spoofed Vault and every partner credential the engine holds.
+
+    Fails closed here rather than in the shared helper, because the error type and cell name are this
+    module's: ``requests`` would otherwise raise deep inside the first KV read, surfacing as an opaque
+    resolution failure that names no cause."""
+    ca = os.environ.get(_ENV_CA_FILE) or None
+    if ca is not None and not os.path.isfile(ca):
+        raise SecretProviderError(
+            f"[secrets].provider={_EXTRA!r}: {_ENV_CA_FILE} names {ca!r}, which is not a readable "
+            f"file — point it at the PEM of the CA that issued the Vault server certificate."
+        )
+    return vault_client_verify_kwargs(ca_file=ca, addr=addr, cell=f"[secrets].provider={_EXTRA!r}")
+
+
 def _build_client(addr: str | None, token: str | None) -> Any:
     """Construct an ``hvac.Client``. Factored out so tests can substitute a fake KV backend without a live
     Vault. ``addr``/``token`` pass through; when ``None``, hvac falls back to its own VAULT_ADDR/VAULT_TOKEN
@@ -77,7 +105,11 @@ def _build_client(addr: str | None, token: str | None) -> Any:
     # egress does. `token` rides as an `X-Vault-Token` header on every KV read, so a 3xx from an
     # on-path attacker (absent TLS integrity) or a spoofed Vault would otherwise relocate the
     # request carrying it. See store/keyprovider_vault.py's twin for the measurement.
-    client: Any = hvac.Client(url=addr, token=token, allow_redirects=False)
+    # #1180 (ASVS 12.3.4): narrow the trust anchor when the operator named one. The keyword is OMITTED
+    # when they did not, so the stock construction is unchanged rather than passed an explicit default.
+    client: Any = hvac.Client(
+        url=addr, token=token, allow_redirects=False, **_vault_ca_kwargs(addr)
+    )
     return client