Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .expo-agents/code-review/config.jsonc
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,15 @@
"timeoutMs": 30000
},

// Active Jev verification cascade. Confident local judgments act immediately;
// ambiguous and protected findings defer to the reasoning verifier.
"jev": {
"model": "jev-1.13.0",
"minConfidence": 0.9,
"timeoutMs": 10000,
"maxContextChars": 30000
},

// Large diffs are split into focused chunks by changed-line count, plus a
// cross-cutting pass for multi-file issues. Diffs under maxChangedLines are one
// full-context pass. Defaults shown; raise/lower per your model + PR sizes.
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/expo-code-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -90,5 +90,7 @@ jobs:
CLAUDE_CODE_REVIEW_SHARED_API_TOKEN: ${{ secrets.CLAUDE_CODE_REVIEW_SHARED_API_TOKEN }}
# Search-only credential used for fixed, site-scoped documentation discovery.
BRAVE_SEARCH_API_KEY: ${{ secrets.BRAVE_SEARCH_API_KEY }}
# Shadow-only Jev evaluator; unused while config.jev.enabled is false.
TYPESAFE_API_KEY: ${{ secrets.TYPESAFE_API_KEY }}
# Optional: override the model for every agent.
REVIEWER_MODEL: ${{ vars.REVIEWER_MODEL }}
33 changes: 31 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,35 @@ The boundary, in brief:
Full detail — providers, query grammar, `fetch_platform_doc` modes, provenance and
citation grounding: [LLP 0013](./llp/0013-platform-research.explainer.md).

## Jev selective verification

ECR can use [TypeSafe AI's Jev](https://docs.typesafe.ai/) as the fast first stage of
finding verification. Generative agents still discover and explain problems. Jev
answers one narrow typed question against bounded local source, then either keeps a
clearly supported finding, drops a clearly contradicted ordinary finding, or defers
to the existing reasoning verifier.

There is no shadow mode or separate `enabled` flag. Add the root-only block and provide
`TYPESAFE_API_KEY`; removing the block restores the original verification path:

```jsonc
{
"jev": {
"model": "jev-1.13.0",
"minConfidence": 0.9,
"timeoutMs": 10000,
"maxContextChars": 30000
}
}
```

Each request contains one finding and bounded source from its file. Critical, security,
secrets, and cited findings are never dropped by Jev alone. Low confidence, missing
context, credentials, timeouts, and provider errors fall back to the reasoning or
original verification path. Confidence is distribution concentration, not proof, so
the model and threshold are explicit. See [LLP 0014](./llp/0014-jev-selective-verification.explainer.md)
for the research, decision table, and threat model.

## Monorepos (routing manifest)

A monorepo can route different subtrees to different reviewer rosters from a single
Expand Down Expand Up @@ -355,9 +384,9 @@ your-monorepo/

Enforced in code and by an independent CI guard step, not by convention:

- **auth and research are locked to the root.** `tokenEnv` is honored only in the
- **auth, research, and Jev are locked to the root.** `tokenEnv` is honored only in the
root `config.jsonc` / `routing.jsonc` `defaults.auth`; a scope config declaring
`auth`/`breakGlass`/`research` fails to parse, and the CI guard sweeps every
`auth`/`breakGlass`/`research`/`jev` fails to parse, and the CI guard sweeps every
config file repo-wide and refuses to run unless `tokenEnv` appears exactly once,
root-owned, equal to `ECR_EXPECTED_TOKEN_ENV`. Routing globs choose *which
roster* reviews a file, never *which secret* is sent.
Expand Down
3 changes: 3 additions & 0 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion llp/0000-expo-code-review-cli.explainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
**Author:** Philippe Loulidi / Claude
**Date:** 2026-07-30
**Role:** Root
**Related:** [LLP 0001](0001-trust-model.principles.md), [LLP 0002](0002-review-engine-pipeline.explainer.md), [LLP 0003](0003-model-runtimes-and-credentials.explainer.md), [LLP 0004](0004-diff-noise-and-prompts.explainer.md), [LLP 0005](0005-verification-fingerprints-rendering.explainer.md), [LLP 0006](0006-config-schema-loading-routing.explainer.md), [LLP 0007](0007-cli-commands-and-ci.explainer.md), [LLP 0008](0008-sources-and-reporters.explainer.md), [LLP 0009](0009-adoption-templates-and-ci-workflows.guide.md), [LLP 0013](0013-platform-research.explainer.md)
**Related:** [LLP 0001](0001-trust-model.principles.md), [LLP 0002](0002-review-engine-pipeline.explainer.md), [LLP 0003](0003-model-runtimes-and-credentials.explainer.md), [LLP 0004](0004-diff-noise-and-prompts.explainer.md), [LLP 0005](0005-verification-fingerprints-rendering.explainer.md), [LLP 0006](0006-config-schema-loading-routing.explainer.md), [LLP 0007](0007-cli-commands-and-ci.explainer.md), [LLP 0008](0008-sources-and-reporters.explainer.md), [LLP 0009](0009-adoption-templates-and-ci-workflows.guide.md), [LLP 0013](0013-platform-research.explainer.md), [LLP 0014](0014-jev-selective-verification.explainer.md)

This is the root map for `@expo/code-review-cli` (`ecr`). It records what the system does, how the subsystems divide, the invariants that hold across all of them, and where each concern's rationale lives. It is intentionally thin: every subsystem's detail belongs in the doc that owns it. Read this first, then follow the link for the part you care about.

Expand All @@ -29,6 +29,7 @@ The code divides by concern. Each directory maps to the doc that owns its ration
- `src/sources/` (`local-git`, `github-pr`) + `src/reporters/` (`terminal`, `github`) — where the diff comes from and where findings go [observed: AGENTS.md:14-15; src/sources/, src/reporters/]. Owned by [LLP 0008](0008-sources-and-reporters.explainer.md).
- `templates/` + the scaffolded CI workflow — the files `ecr init` writes into an adopting repo; they define adopter-facing behavior and must stay in sync with the code [observed: AGENTS.md:19-20]. Owned by [LLP 0009](0009-adoption-templates-and-ci-workflows.guide.md).
- `src/research-mcp/` + `research/` — the bundled read-only documentation MCP, scoped remote discovery and allowlisted fetch adapters, optional index builder, source catalog, and host-side evidence prepass. Owned by [LLP 0013](0013-platform-research.explainer.md).
- `src/core/jev.ts` — optional typed first stage of finding verification against bounded local source. Owned by [LLP 0014](0014-jev-selective-verification.explainer.md).
- The trust model spans all of the above and is stated as principles in [LLP 0001](0001-trust-model.principles.md).

## Cross-Cutting Invariants
Expand Down Expand Up @@ -70,5 +71,6 @@ Start here, then follow the owning doc.
| What do the CLI commands do, and how does CI orchestrate them? | [LLP 0007](0007-cli-commands-and-ci.explainer.md) | `src/cli.ts`, `src/commands/*` |
| Where does the diff come from and where do findings go? | [LLP 0008](0008-sources-and-reporters.explainer.md) | `src/sources/*`, `src/reporters/*` |
| What does `ecr init` scaffold into an adopting repo? | [LLP 0009](0009-adoption-templates-and-ci-workflows.guide.md) | `templates/`, scaffolded CI workflow |
| How does Jev make narrow decisions and defer uncertain findings? | [LLP 0014](0014-jev-selective-verification.explainer.md) | `src/core/jev.ts`, `src/core/verify.ts` |

Non-doc sources of truth: `AGENTS.md` holds the working conventions (import style, error handling, the `opencode-ai`/`@opencode-ai/sdk` pinning rule, security invariants); `ROADMAP.md` lists open and deferred items — incremental review, inline PR comments, result-level caching, and a Codex token rotator among them are still open and must not be assumed implemented [observed: ROADMAP.md]; and `templates/` is the adopter-facing contract, so treat it as behavior, not sample text.
4 changes: 2 additions & 2 deletions llp/0002-review-engine-pipeline.explainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

## Pipeline Stages

`runReview()` runs a fixed sequence: filter noise → resolve the read root (the trust boundary) → start the engines → preflight that every model id resolves → chunk the diff → run agent×chunk passes plus one cross-cutting pass through a growable bounded queue → coordinate → verify → suppress → reconcile the summary → log. The whole run's resource lifecycle — engine handles, auth material, cwd, and the read-root worktree — is released in a single `finally` (`review.ts:947-951`), so no partial-startup path leaks a server or a worktree.
`runReview()` runs a fixed sequence: filter noise → resolve the read root (the trust boundary) → start the engines → preflight that every model id resolves → chunk the diff → run agent×chunk passes plus one cross-cutting pass through a growable bounded queue → coordinate → verify → suppress → reconcile the summary → log. Optional Jev classification is the first stage inside verification: clear local judgments act and uncertainty defers to the reasoning verifier. [LLP 0014](0014-jev-selective-verification.explainer.md) owns that boundary. The whole run's resource lifecycle — engine handles, auth material, cwd, and the read-root worktree — is released in a single `finally` (`review.ts:947-951`), so no partial-startup path leaks a server or a worktree.

Two ordering decisions inside that lifecycle are load-bearing and easy to break by "cleaning up":

Expand Down Expand Up @@ -77,7 +77,7 @@ Rate-limit evidence is tracked per engine separately and surfaced with the cause

`writeRunLog()` (`src/core/log.ts`) appends one JSON line per run — cost, tokens (run-wide and per bucket), findings, coverage notes, decision — so runs are auditable and cost/latency measurable later. The record deliberately excludes PR title and body text, keeping only `baseRef`/`headRef`, to avoid persisting secrets that might appear in author-controlled text [observed] (`log.ts:13-18`). Findings may quote changed source lines, but only content the review already publishes verbatim in the PR comment.

Each successful reviewer pass may also return a bounded conclusion-only trace: up to three concrete checks and two unresolved questions, with each note capped at 240 characters. Trace parsing is deliberately fail-soft: malformed optional diagnostics are dropped while valid findings from the same response survive, so observability can never create a coverage gap. `runReview()` merges and de-duplicates valid notes per reviewer, sorts reviewer ids, caps both the stored roster and the complete decoded trace (6 KB), labels the result `unverified-model-diagnostics`, and attaches it to `CoordinatorOutput.reviewTrace` only after the decision pipeline finishes. `attachReviewTrace` strips any coordinator-authored value before attaching the engine-built trace; even a malformed injected coordinator value parses as absent rather than failing consolidation. The trace therefore never reaches coordination, verification, suppression, or decision logic. Because the GitHub comment already embeds the full coordinator output as durable state, downstream agents can recover the trace without adding visible review prose or storing raw transcripts and chain-of-thought. The same trace rides the run log for short-lived operational inspection [observed: `schema.ts` `ReviewerTraceNotesSchema`/`ReviewTraceSchema`/`ReviewerOutputSchema`; `review.ts` `mergeTraceNotes`/`buildReviewTrace`/`attachReviewTrace`; `render.ts` embeds the full review].
Each successful reviewer pass may also return a bounded conclusion-only trace: up to three concrete checks and two unresolved questions, with each note capped at 240 characters. Trace parsing is deliberately fail-soft: malformed optional diagnostics are dropped while valid findings from the same response survive, so observability can never create a coverage gap. `runReview()` merges and de-duplicates valid notes per reviewer, sorts reviewer ids, caps both the stored roster and the complete decoded trace (6 KB), labels the result `unverified-model-diagnostics`, and attaches it to `CoordinatorOutput.reviewTrace` only after the decision pipeline finishes. `attachReviewTrace` strips any coordinator-authored value before attaching the engine-built trace; even a malformed injected coordinator value parses as absent rather than failing consolidation. The trace therefore never reaches coordination, verification, suppression, or decision logic. Because the GitHub comment already embeds the full coordinator output as durable state, downstream agents can recover the trace without adding visible review prose or storing raw transcripts and chain-of-thought. The same trace rides the run log for short-lived operational inspection [observed: `schema.ts` `ReviewerTraceNotesSchema`/`ReviewTraceSchema`/`ReviewerOutputSchema`; `review.ts` `mergeTraceNotes`/`buildReviewTrace`/`attachReviewTrace`; `render.ts` embeds the full review]. Optional Jev aggregate usage is a second log sink; individual suppressions use the ordinary `verifierDropped` audit trail, and neither contains extra source text.

Observability must never break a review, so both output sinks swallow their own errors: `safeLog` wraps `writeRunLog` and discards any write error [observed] (`review.ts:1269-1275`), and `appendStepSummary` is a no-op outside Actions and swallows append failures [observed] (`step-summary.ts:9-19`). A consequence worth stating: a missing or corrupt run-log entry is therefore **not** an error signal — the log's presence or absence proves nothing about whether the review succeeded.

Expand Down
10 changes: 9 additions & 1 deletion llp/0006-config-schema-loading-routing.explainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ caller, [LLP 0007](0007-cli-commands-and-ci.explainer.md) and [LLP 0001](0001-tr
## Root vs Scope Config

A scope config is the root schema minus the centrally locked keys. `ScopeReviewConfigSchema`
takes `ReviewConfigSchema`, omits `auth`, `breakGlass`, and `commentTag`, and re-adds each as
takes `ReviewConfigSchema`, omits centrally owned keys including `auth`, `breakGlass`,
`commentTag`, `research`, and `jev`, and re-adds each as
`z.never().optional()` so declaring any of them is a hard parse error, not a later runtime
check [observed: `schema.ts:232-247`]. The point of failing at the Zod level is that an IDE or
`doctor` catches it before CI does [observed: `schema.ts:224-225` comment]. The remaining
Expand Down Expand Up @@ -84,6 +85,13 @@ commit `43b31a4` follow-up]. The actual secret values never enter this subsystem
owns the wiring and the `FORBIDDEN_TOKEN_ENVS` list, and schema/load only ever touch the env-var
NAME [observed: `AGENTS.md` security invariants; `src/core/auth.ts:28,45,206`].

`jev` is root-only for the same trust reason as research: adding it sends bounded
repository-derived text to another provider. A scope may neither activate it nor alter
its model, threshold, or limits. Nested scopes inherit the root's resolved settings, so
one PR has one centrally owned disclosure policy [observed: `schema.ts`
`ScopeReviewConfigSchema`; `load.ts` `loadScopeConfig`]. The data boundary and active
selective cascade live in [LLP 0014](0014-jev-selective-verification.explainer.md).

## Model Resolution

The `REVIEWER_MODEL` env override is applied on top of frontmatter/default model resolution using
Expand Down
9 changes: 9 additions & 0 deletions llp/0009-adoption-templates-and-ci-workflows.guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,15 @@ credential actually selected by trusted config. The default remains Anthropic an
`dismiss.yml` still receives no model credential [observed: `templates/config.jsonc`;
`templates/workflow.yml`, `command.yml`, and `atlantis.yml`].

**Jev is an independent optional secret.** Each review-running workflow maps
`TYPESAFE_API_KEY`; adding `jev` to trusted root config activates the selective cascade.
GitHub supplies an empty value when the repository secret is absent, so the original
verification path remains available. ECR sends bounded finding/file source directly
to TypeSafe and logs aggregate usage; the key never enters `auth.tokenEnv` or reviewer
engines. `dismiss.yml` still receives no model or Jev credential.
[LLP 0014](0014-jev-selective-verification.explainer.md) owns the disclosure and
fallback rules.

**Keep the `**/*` catch-all first in `routing.jsonc`.** Scopes match last-match-wins, so a `**/*` catch-all must come first and more specific scopes come after to override it [observed: `templates/routing.jsonc:27-31`; `AGENTS.md:17-18`]. Nothing validates the order — reversing it silently changes which config applies to already-routed files [inferred: no code checks that the `**/*` catch-all comes first, so reversing it silently changes routing].

**`scope-config.jsonc` deliberately omits `auth` and `commentTag`.** It is a *different* template from the root `config.jsonc`, not a copy. A scope config carries no `auth` block — credentials are root-only, and the loader plus the CI guard reject a scope config that declares one [observed: `templates/scope-config.jsonc:1-3`; `AGENTS.md:51`]. It also omits `commentTag`, because a scope's PR-comment marker is always derived as `<rootTag>:<scope-name>` so `ecr ci` and `ecr review --scope --post` target the same comment; declaring one is rejected by the scope schema [observed: `templates/scope-config.jsonc:23-25`]. Reusing the root `config.jsonc` template for a scope — a tempting "simplification" — would break the trust boundary (LLP 0006).
Expand Down
Loading
Loading