Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,30 @@ Streamable HTTP; the v0.3 `guard` proxy adds deterministic runtime *result* insp

## [Unreleased]

### Added

- **`deploy-gate` — fail-closed CI gate for agent deployments (DSE-1257).** Verifies a
deploy's evidence against a declared gate policy: required eval suites met their
thresholds, required guardrails are active, a budget/quota is declared, and a human
approval receipt is present when required. The gate **adjudicates evidence rather than
running evals** — keeping verdicts reproducible from two JSON files, free of any eval
framework's dependency tree, and making missing/malformed evidence an unambiguous
failure instead of a silent skip. Nine `WRD-GATE-*` rules; exit `0` only when every
control is satisfied, `1` on any finding, `2` on unreadable input (fail closed).
See [`docs/AGENT_GATES.md`](docs/AGENT_GATES.md).
- **`auth audit` — static MCP auth-posture audit (DSE-1258).** Audits MCP client/server
config for remote endpoints declaring no authentication, cleartext `http://` transport,
and credential literals committed into config (`WRD-AUTH-*`), reusing the existing
vendor secret patterns from `check`. **Static only** — no server spawn, no DNS, no
network — which keeps it safe to run against any config in CI and immune to churn in
the MCP auth specification. Deliberately does not flag loopback servers, `${VAR}`
secret references, or local stdio servers; every credential literal is redacted in
findings, snippets, and SARIF. Runtime capability brokering stays out of scope
(DSE-725). See [`docs/AGENT_GATES.md`](docs/AGENT_GATES.md).

Both commands reuse the `check` exit-code contract and the shared SARIF/JSONL emitters,
so an existing code-scanning pipeline needs no changes.

## [1.1.0] — 2026-07-14

### Added
Expand Down
18 changes: 18 additions & 0 deletions DOCUMENTATION_INDEX.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,24 @@ describe and visualize the implementation that satisfies that contract.
| 2 | [`SYSTEM_CONTEXT_DIAGRAM.md`](SYSTEM_CONTEXT_DIAGRAM.md) | System context + pin/check sequence (mermaid); trust boundary; `conclave` as dev-time reviewer only; composite GitHub Action + **pre-commit hook** as consumer delivery vehicles |
| 3 | [`DOCUMENTATION_INDEX.md`](DOCUMENTATION_INDEX.md) | This file |

## Agent gates (`deploy-gate` + `auth audit` — DSE-1257 / DSE-1258)

Two fail-closed gates extending warden past MCP-surface integrity into the adjacent
CI controls agent deployments lack: did the deploy meet its declared safety bar, and
is the MCP auth posture sound. Both reuse the `check` exit-code contract (0 clean /
1 finding / 2 fail-closed) and the shared SARIF + JSONL emitters.

| Artifact | Purpose |
|----------|---------|
| [`docs/AGENT_GATES.md`](docs/AGENT_GATES.md) | Security contract for both gates — policy/evidence schemas, full rule tables, scope honesty, and the two load-bearing design decisions (evidence-adjudication, static-only) |
| [`src/mcp_warden/deploy_gate.py`](src/mcp_warden/deploy_gate.py) | `WRD-GATE-*` engine: eval thresholds, guardrail presence, budget, approval receipt |
| [`src/mcp_warden/cli_deploy_gate.py`](src/mcp_warden/cli_deploy_gate.py) | `deploy-gate` command body (register idiom) |
| [`src/mcp_warden/auth_audit.py`](src/mcp_warden/auth_audit.py) | `WRD-AUTH-*` static config audit; reuses `checks_secret.scan_field` for vendor patterns |
| [`src/mcp_warden/cli_auth.py`](src/mcp_warden/cli_auth.py) | `auth audit` sub-app command body |
| [`tests/test_deploy_gate.py`](tests/test_deploy_gate.py) | Engine per-control pass/fail + CLI exit codes + fail-closed on malformed evidence |
| [`tests/test_auth_audit.py`](tests/test_auth_audit.py) | Every rule, the deliberate non-flags (loopback, `${VAR}` refs, stdio), redaction, CLI/JSON/SARIF |
| [`examples/agent-gates/`](examples/agent-gates/) | Runnable demos — a 6-server config (4 flagged / 2 deliberately clean) and pass/fail deploy evidence, with expected verdicts documented and verified |

## GitHub Action (`action.yml` — Issue #18)

The composite reusable action is the primary delivery vehicle for the `check` gate.
Expand Down
20 changes: 19 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,10 +107,23 @@ closes gaps none of them cover alone.
| **Static tool-poisoning scanner** | [mcp-scan](https://github.com/invariantlabs-ai/mcp-scan) | pin-time / pre-flight | suspicious *content* in tool definitions (injection-style descriptions, known-bad patterns) | you want to catch a poisoned definition the first time you see it |
| **Runtime gateway / proxy** | ContextForge, Lunar MCPX, TrueFoundry, Docker MCP Gateway | every live request | runtime mediation — auth, rate limits, request/response policy on calls in flight | you need to mediate or police live traffic between agent and server |
| **Lockfile + CI gate** | **mcp-warden** | CI / pre-commit | *drift* — the declared surface changing after a human approved it (rug-pull / silent redefinition) | you want a reproducible, human-approved baseline that fails the build when the surface changes |
| **Config + deploy gates** | **mcp-warden** (`auth audit`, `deploy-gate`) | CI / pre-commit | *posture* — remote MCP endpoints with no auth or credentials pasted into config; agent deploys whose evals regressed or whose guardrails were switched off | you want the deploy blocked, not just reported, when the declared safety bar isn't met |

mcp-warden does not replace a scanner or a gateway — it adds the missing **drift
gate**: a signed baseline plus a deterministic CI check that the surface you
approved is the surface you still run. For the full, sourced breakdown of how
approved is the surface you still run.

**The common thread across all four commands is that they *block*.** The loudest
complaint about agents in production is that they are insecure by default and
nothing stops a bad configuration or a regressed deploy from shipping — plenty of
tools *report*, very few return a non-zero exit code that a pipeline must answer
for. `check` blocks on surface drift, `auth audit` blocks on weak MCP auth
posture, and `deploy-gate` blocks an agent deploy whose evals, guardrails,
budget, or human approval don't meet the declared bar. All three fail **closed**:
unreadable or missing input is a failure, never a silent pass. See
[`docs/AGENT_GATES.md`](docs/AGENT_GATES.md).

For the full, sourced breakdown of how
these layers complement each other and when to use which, see the
[**comparison page**](https://datascience-engineeringexperts.github.io/mcp-warden/comparison/)
on the docs site.
Expand All @@ -129,6 +142,9 @@ automatically — so the use cases are sequenced by leverage:
pre-commit hook) fails when upstream silently redefines its surface — the core rug-pull defense.
- **Security / platform engineer.** Run the [Action](#github-action-one-step-drop-in)
across a fleet; SARIF → code scanning; signed locks = auditable human-approval evidence.
Add `auth audit` to catch MCP endpoints configured without auth or with credentials
committed into config, and `deploy-gate` to make the agent safety bar a build failure
rather than a dashboard nobody reads.
- **Incident responder / auditor.** `inspect` an offline trace and `warden diff` a suspect
lock against a known-good baseline — no live server required.
- **Agent-framework integrator** *(post-launch).* Enforce that only warden-locked servers
Expand Down Expand Up @@ -391,6 +407,8 @@ run the gate only on push:
| `mcp-warden inspect <trace.jsonl> [--lock F] [--sarif F]` | **(v0.2)** Offline analyzer over a recorded JSON-RPC session — same `WRD-RES-*` catalog as `guard` (always report-only) | non-zero on any BLOCK-tier finding; 2 on read error |
| `mcp-warden lock rotate <lock> [--approver ID] [--actor ID] [--note T] [--json]` | **(v0.3)** Re-attest provenance on an existing baseline without re-capturing the surface; `overall_digest` stays **byte-identical** (WARDEN_LOCK_SCHEMA §8.2). Fails closed on a tampered/inconsistent lock | 0 on success, 2 on missing/invalid/tampered lock |
| `mcp-warden diff <lock-a> <lock-b> [--json] [--sarif F] [--no-provenance] [--exit-code]` | **(v0.3)** Offline, **redacted** viewer over the drift engine: renders integrity drift between two existing locks (A=baseline, B=current) + a separate informational provenance section. Never re-captures and never prints raw `server.command`/`args` (secret-safe) | 0 (viewer); with `--exit-code`, 1 on **integrity** drift only; 2 on missing/invalid lock |
| `mcp-warden deploy-gate --policy F --evidence F [--json] [--sarif F]` | **(v1.2)** Fail-closed CI gate for agent deployments: verifies declared eval thresholds, required guardrails, a budget/quota, and a human-approval receipt. Adjudicates evidence — it does **not** run evals. See [`docs/AGENT_GATES.md`](docs/AGENT_GATES.md) | 0 only when every control is satisfied; 1 on any gate finding; 2 on unreadable/malformed input (fail closed) |
| `mcp-warden auth audit <config...> [--json] [--sarif F]` | **(v1.2)** Static MCP auth-posture audit over client/server config: remote endpoints without auth, cleartext `http://`, credential literals committed into config. No server spawn, no network. See [`docs/AGENT_GATES.md`](docs/AGENT_GATES.md) | 0 clean; 1 on any finding; 2 on read/parse error (fail closed) |
| `mcp-warden-precommit [--lock F] [--timeout N] [--strict] -- <server-cmd...>` | **(v0.3)** pre-commit hook entry point (see [pre-commit hook](#pre-commit-hook--the-local-pre-ci-gate)). Runs the same check verdict path; check-only (never pins, never writes the lock) | 0 clean / **1 drift** / 2 config error; server-unavailable → 0+warning (non-strict) or 2 (`--strict`) |

For stdio, `<server-cmd...>` is passed to the OS as an **argv array, never through a
Expand Down
11 changes: 11 additions & 0 deletions SYSTEM_CONTEXT_DIAGRAM.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,17 @@ logic) plus a separate informational provenance section. It never prints raw
> is a **dev-time design reviewer** that shaped this contract. It is **NOT** a
> runtime dependency and is never invoked by `pin`/`check`/`policy`.

> **Agent gates (DSE-1257 / DSE-1258)** add two CI-only verbs that sit alongside `check`
> and never touch a running server. `deploy-gate` reads two JSON documents (a gate policy
> and pipeline-produced deploy evidence) and fail-closes a deploy whose declared eval
> thresholds, guardrails, budget, or human-approval receipt are unmet — it adjudicates
> evidence and deliberately does **not** execute evals. `auth audit` reads MCP client
> config files and flags weak auth posture statically: no server spawn, no DNS, no
> network, which is what keeps it immune to MCP auth-spec churn. Both reuse the `check`
> exit-code contract (0/1/2, fail closed) and the shared SARIF + JSONL emitters, so they
> enter an existing code-scanning pipeline with no new plumbing. Runtime capability
> brokering remains out of scope (DSE-725). See [`docs/AGENT_GATES.md`](docs/AGENT_GATES.md).
>
> **`action.yml` (Issue #18)** is the primary consumer delivery vehicle for the `check`
> gate. Consumers pin `DataScience-EngineeringExperts/mcp-warden@<tag>` in their workflow; the composite
> action wraps the C2 sequence (steps 1–5 of the pin/check sequence above) behind a
Expand Down
136 changes: 136 additions & 0 deletions docs/AGENT_GATES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
# Agent Gates — `deploy-gate` and `auth audit`

Two fail-closed gates that extend mcp-warden past MCP-surface integrity into the
two adjacent controls that agent deployments actually lack in CI: **did the
deploy meet its declared safety bar** (`deploy-gate`, DSE-1257) and **is the MCP
auth posture sound** (`auth audit`, DSE-1258).

Both follow the same contract as `check`:

| Exit | Meaning |
|------|---------|
| `0` | Every declared control satisfied |
| `1` | At least one finding — the gate blocks |
| `2` | Unreadable/malformed input — **fail closed**, never a pass |

Both emit `--json` (JSONL findings) and `--sarif` (code-scanning upload), reusing
the same emitters as `check`, so an existing SARIF pipeline needs no changes.

---

## 1. `deploy-gate` — release control for agent deployments

**The gap.** Agent frameworks ship evals, guardrails, and budgets as libraries,
but nothing *blocks a deploy* when the evals regress or a guardrail is switched
off. Teams write bespoke shell in CI, or skip the check.

**The design decision that matters: the gate does not run evals.** It verifies
*evidence* that they ran and passed. Running evals is the pipeline's job and is
framework-specific; adjudicating them is a deterministic, portable control. This
keeps the gate free of every eval framework's dependency tree, makes verdicts
reproducible from two JSON files, and means a missing or malformed evidence file
is unambiguously a **failure** rather than a silent skip.

### Policy schema

```json
{
"required_evals": [{ "suite": "safety", "min_score": 0.9 }],
"required_guardrails": ["prompt-injection", "pii-redaction"],
"require_budget": true,
"require_approval": true
}
```

### Evidence schema

Produced by the deploy pipeline:

```json
{
"evals": { "safety": { "score": 0.95 } },
"guardrails": ["prompt-injection", "pii-redaction"],
"budget": { "limit": 100 },
"approval": { "approved": true, "approver": "release-manager@example.com" }
}
```

### Rules

| Rule ID | Severity | Fires when |
|---------|----------|-----------|
| `WRD-GATE-EVAL-MISSING` | high | A required suite has no result in evidence |
| `WRD-GATE-EVAL-MALFORMED` | high | A suite reported no numeric score |
| `WRD-GATE-EVAL-THRESHOLD` | high | A suite scored below its `min_score` |
| `WRD-GATE-EVAL-EVIDENCE` | high | The `evals` block is not an object |
| `WRD-GATE-GUARDRAIL-MISSING` | high | A required guardrail is not active |
| `WRD-GATE-BUDGET-MISSING` | medium | `require_budget` set, no budget declared |
| `WRD-GATE-BUDGET-INVALID` | medium | Budget has no positive limit |
| `WRD-GATE-APPROVAL-MISSING` | critical | `require_approval` set, no receipt |
| `WRD-GATE-APPROVAL-INVALID` | critical | Receipt is not affirmative and attributed |

### Usage

```bash
mcp-warden deploy-gate --policy gate-policy.json --evidence deploy-evidence.json
```

```yaml
- name: Agent deploy gate
run: mcp-warden deploy-gate --policy gate-policy.json --evidence evidence.json --sarif gate.sarif
```

### Scope honesty

`deploy-gate` adjudicates **declared evidence**. It does not verify that the
evidence is truthful — a pipeline that fabricates a score passes. Bind evidence
to a trusted producer (signed CI artifact, restricted branch) when that matters.
It is a release control, not an attestation system; the signed-decision path
lives in [`POLICY_ENFORCEMENT.md`](POLICY_ENFORCEMENT.md).

---

## 2. `auth audit` — static MCP auth-posture audit

**The gap.** MCP configs routinely point at remote endpoints with no
authentication, over cleartext `http://`, with bearer tokens pasted directly
into the committed config. None of that requires exploitation to find — it is
declared in the file.

**The design decision that matters: static only.** No server is spawned, no DNS
is resolved, no network is touched. The audit reasons purely about what the
config declares. That makes it safe to run against any config in CI, and — this
is the load-bearing part — it keeps the feature **immune to churn in the MCP
auth specification**. Runtime capability brokering is deliberately out of scope
and tracked separately (DSE-725).

### Rules

| Rule ID | Severity | Fires when |
|---------|----------|-----------|
| `WRD-AUTH-NOAUTH` | medium | A remote endpoint declares no auth material |
| `WRD-AUTH-PLAINTEXT-HTTP` | high | A remote endpoint uses `http://` |
| `WRD-AUTH-TOKEN-IN-CONFIG` | high | An auth-bearing key holds a literal credential |
| `WRD-AUTH-URL-CREDENTIAL` | high | The endpoint URL embeds a `user:pass@` userinfo credential |
| `WRD-SEC-*` | varies | Vendor secret patterns found in any config value (shared with `check`) |

### What it deliberately does not flag

Precision matters more than recall for a gate that blocks CI:

- **Loopback servers** (`localhost`, `127.0.0.1`, `::1`) — not remotely
reachable, so missing auth is not an exposure.
- **Secret references** — `${TOKEN}`, `$TOKEN`, `{{ secret }}` are the correct
pattern and are never flagged as literals.
- **Local stdio servers** — a `command`/`args` entry with no URL and no remote
transport has no auth posture to audit.

### Usage

```bash
mcp-warden auth audit ~/.claude/claude_desktop_config.json .mcp.json
mcp-warden auth audit .mcp.json --sarif auth.sarif
```

Every credential literal is redacted in findings, snippets, and SARIF output —
the audit never widens exposure of the thing it is reporting.
1 change: 1 addition & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ are re-checked on every run so these examples stay green.
| [`gitlab-ci/.gitlab-ci.yml`](gitlab-ci/.gitlab-ci.yml) | the same check gate on GitLab CI |
| [`pre-commit/.pre-commit-config.yaml`](pre-commit/.pre-commit-config.yaml) | local pre-commit + pre-push hook variants |
| [`pinned-servers/`](pinned-servers/) | real MCP servers pinned to a committed `warden.lock` each |
| [`agent-gates/`](agent-gates/) | runnable `deploy-gate` + `auth audit` demos — a config with four flagged servers and two that must NOT be flagged, plus pass/fail deploy evidence |

## Pinned-server examples

Expand Down
61 changes: 61 additions & 0 deletions examples/agent-gates/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# Agent gates — runnable examples

Two fail-closed CI gates. Full contract: [`docs/AGENT_GATES.md`](../../docs/AGENT_GATES.md).

## `auth audit` — static MCP auth posture

```bash
mcp-warden auth audit examples/agent-gates/mcp-config-audit-demo.json
```

The demo config has six servers. **Four are flagged, two must not be** — the
non-flags matter as much as the findings, because a gate that cries wolf on
correct configuration gets disabled.

| Server | Verdict |
|--------|---------|
| `filesystem-local` | clean — local stdio server, no auth posture to audit |
| `loopback-dev` | clean — loopback is not remotely reachable |
| `good-citizen` | clean — `${VENDOR_TOKEN}` is a reference, not a literal |
| `internal-http` | `WRD-AUTH-PLAINTEXT-HTTP` (high) + `WRD-AUTH-NOAUTH` (medium) |
| `vendor-api` | `WRD-AUTH-TOKEN-IN-CONFIG` (high) + `WRD-SEC-ENTROPY` (high) |
| `legacy` | `WRD-AUTH-URL-CREDENTIAL` (high) + `WRD-AUTH-NOAUTH` (medium) |

Exits `1` with six findings. Every credential is redacted in output and SARIF.

Static only: no server is spawned, no DNS is resolved, no network is touched.

## `deploy-gate` — release control for agent deploys

```bash
mcp-warden deploy-gate \
--policy examples/agent-gates/gate-policy.json \
--evidence examples/agent-gates/evidence-pass.json # exit 0

mcp-warden deploy-gate \
--policy examples/agent-gates/gate-policy.json \
--evidence examples/agent-gates/evidence-fail.json # exit 1
```

`evidence-fail.json` trips four controls at once: the safety eval regressed
below threshold, a required guardrail was switched off, the budget has no
positive limit, and the approval receipt is unattributed.

The gate **adjudicates evidence — it does not run evals.** Your pipeline runs
them and writes the evidence file; the gate decides deterministically whether
the deploy may proceed. Missing or malformed evidence is a failure, never a
pass.

## In CI

```yaml
- name: MCP auth posture
run: mcp-warden auth audit .mcp.json --sarif auth.sarif

- name: Agent deploy gate
run: mcp-warden deploy-gate --policy gate-policy.json --evidence evidence.json --sarif gate.sarif

- uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: auth.sarif
```
10 changes: 10 additions & 0 deletions examples/agent-gates/evidence-fail.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"_comment": "A deploy that must be blocked: safety eval regressed below threshold, the pii-redaction guardrail was switched off, the budget has no positive limit, and the approval receipt is unattributed.",
"evals": {
"safety": { "score": 0.62 },
"task-success": { "score": 0.91 }
},
"guardrails": ["prompt-injection"],
"budget": { "limit": 0 },
"approval": { "approved": true, "approver": "" }
}
Loading
Loading