Skip to content

CONTRACTS.md

Latest commit

 

History

History
345 lines (283 loc) · 16.5 KB

CONTRACTS.md

File metadata and controls

345 lines (283 loc) · 16.5 KB
Raw

corbits-triage — Frozen Build Contracts

This file is the binding contract between build lanes (core, service, CLI, release/docs). Every literal string, exit code, endpoint, schema and template here is a compatibility surface (NFR-12). Do not change anything in this file without director sign-off.

1. Product identity

  • Product and command name: corbits-triage (IR-1). The CLI binary, docs and all examples use this name.
  • Version: single shared constant VERSION = "0.1.0" exported from packages/core/src/version.ts. CLI and service must match exactly or refuse to run (NFR-9, exit code 4).
  • Runtime: Bun >= 1.3, TypeScript, zero runtime npm dependencies. Allowed built-ins: bun:sqlite, Bun.serve, fetch, node:fs, node:path, node:os, node:crypto, node:readline. Dev dependency: typescript only (for tsc --noEmit). Tests: bun test.

2. Repo layout

package.json              # workspaces root; scripts: typecheck, test, gate
tsconfig.json
LICENSE.md GPL-2.0.txt GPLv2-AI-Exception.md   # already present; do not modify
docs/                     # CONTRACTS.md (this file), BUILD_NOTES.md, install.md, upgrade.md, privacy.md
packages/core/            # domain logic, no network I/O except judge provider client
packages/service/         # service entry, admin API, GitHub adapter, polling, reconciliation
packages/cli/             # corbits-triage CLI
fixtures/                 # corpus fixtures, injection cases, calibration samples
release/                  # Dockerfile, image build script, licence-compat check, release gate script

3. Exit codes (CLI, frozen — from stage 4 block-global-flags)

Code Meaning
0 success
1 generic error
2 usage error (unknown verb/flag)
3 service unreachable
4 CLI/service version mismatch
5 action disabled/locked (unattended threshold not met, migration-failed read-only mode)
6 policy lint failure
7 injection release gate failure

4. Global CLI flags (IR-2, frozen)

--api-url <url>      Admin API base URL (default http://127.0.0.1:8787)
--token-file <path>  Admin token file (default ~/.config/corbits-triage/corbits-triage.token)
--config <path>      Config file (default ~/.config/corbits-triage/config.toml)
--json               Machine-readable JSON output
--yes                Skip confirmation prompts
--no-color           Disable ANSI colour

corbits-triage --help must print all six flags and the exit-code table above.

5. Verbs (IR-3, table-verbs, frozen)

Verb Talks to Purpose
status service four dashboard figures + judge identity + unattended standing + worklist URL
pending service unconfirmed verdicts, oldest first
show <pr> service full private verdict record
confirm <pr> service confirm verdict: edit comment, close PR, remove label
withdraw <pr> [--note <text>] service withdraw verdict: edit comment, PR stays open, remove label
triage <pr> [--dry-run] service run triage for one PR; dry-run renders comment, mutates nothing
reconcile service align records with GitHub state changed by hand
elicit [--cap <dur>] [--corpus <path>] local interview → draft policy
policy init|lint|show|clause <id> local policy management; lint exit 6 on violation
calibrate [--holdback <n>] local calibration replay + report
injection run local injection suite; exit 7 + RELEASE GATE: FAIL on any required-case failure
unattended status|enable|disable [--machine-checks|--model-judgement] service unattended close controls; enable exits 5 when locked
export [--out <path>] service export verdict log (JSON lines)
diagnose --out <path> service redacted diagnostic bundle
contributor forget <handle> service delete private verdict records for a contributor; public comments untouched
auth login local store admin token under ~/.config/corbits-triage/
Frozen table-verbs row order (AC-39): verb-status, verb-pending, verb-show,
verb-confirm, verb-withdraw, verb-triage, verb-reconcile, verb-elicit,
verb-policy, verb-calibrate, verb-injection, verb-unattended, verb-export,
verb-diagnose, verb-forget. auth login is documented in setup, not a
table-verbs row.

6. Admin API (NFR-5, IR-12, frozen)

  • Bind: 127.0.0.1:8787 only. Default polling mode opens no public inbound socket.
  • Auth: Authorization: Bearer <token>. Token generated on first service start, written to the data volume; auth login copies it to the local token path.
  • Version check: every request carries X-Corbits-Triage-Version: <cli version>. Mismatch → 409 with error code version_mismatch. CLI refuses to run (exit 4) and changes no state.
  • Non-loopback remote address → 403 code not_loopback.
  • Missing/bad token → 401 code bad_token.
  • Error shape (all errors, frozen):
{ "error": { "code": "not_loopback", "message": "human sentence", "posted": false, "closed": false, "next": "corbits-triage <verb>" } }

posted/closed state whether anything was posted or closed (IR-9). next is omitted when there is no next command.

Endpoints (all under /v1, JSON):

Method + path Purpose
GET /v1/status {version, queue_depth, oldest_queued_age_s, pending_count, oldest_pending_age_s, judge:{model_id,prompt_version,policy_version}, unattended:{machine_checks:"on"|"off", model_judgement:"locked"|"on"|"off", threshold:{metric:"agreement",required:number,samples_required:number}, standing:{agreement:number|null,samples:number}}, degraded:boolean, worklist_url, overflow_mode, manual_close_mode}
GET /v1/pending {items:[{pr, title, author, age_s, clause_id, confidence, attribution:"machine"|"model"}]} oldest first
GET /v1/verdicts/:pr full private record (see §9)
POST /v1/verdicts/:pr/confirm performs FR-16 sequence, returns ordered effects
POST /v1/verdicts/:pr/withdraw body {note?}; FR-17 sequence
POST /v1/triage/:pr body {dry_run:boolean}; dry-run returns {rendered_comment, would_label, would_close} and performs no GitHub mutation
POST /v1/reconcile runs reconciliation once, returns {changes:[...]}
GET /v1/export JSON-lines verdict log
POST /v1/contributors/:handle/forget deletes private records, returns count
GET /v1/diagnose redacted bundle JSON (no secrets, no private reasoning)
GET /v1/unattended / POST /v1/unattended body {scope:"machine-checks"|"model-judgement", enabled:boolean}; locked model-judgement enable → 409 code unattended_locked with {threshold, standing} (CLI exit 5)

7. Labels (frozen, NFR-12)

  • triage: unconfirmed verdict — pending confirmation
  • triage: needs your read — routed to maintainer (invalid judge result, provider failure)
  • triage: passed — above bar

Worklist URL shape: https://github.com/<owner>/<repo>/pulls?q=is%3Apr+is%3Aopen+label%3A%22triage%3A+unconfirmed+verdict%22

8. Public comment templates (IR-8, NFR-7, frozen)

Rendered only from validated data fields; model prose never enters public text except the quoted clause text (taken mechanically from the policy file by id) and the diff anchor. No confidence, model id, reasoning or rejected alternatives ever appear in public comments.

Marker line (first line, all four states): <!-- corbits-triage:<state>:<verdict-id> --> States: ack, unconfirmed, withdrawn, confirmed.

Acknowledgement (block-comment-acknowledgement):

<!-- corbits-triage:ack:<id> -->
This pull request has entered the maintainer's triage queue for this repository.

You may or may not get a further automated comment. The maintainer reviews triage results privately.

Unconfirmed verdict (block-comment-unconfirmed):

<!-- corbits-triage:unconfirmed:<id> -->
**Triage verdict: below bar.**

This pull request does not meet the repository's contribution policy.

Policy clause `<clause-id>`:

> <clause text quoted mechanically from the policy file>

Where: `<file>:<start>-<end>`

This verdict is unconfirmed as of yet

The maintainer will confirm or withdraw this verdict. If it is confirmed, this pull request will be closed.

The line This verdict is unconfirmed as of yet must appear exactly (FR-11).

Confirmed verdict (block-comment-confirmed): same comment edited — marker state becomes confirmed, the unconfirmed line and trailing sentence are replaced by:

The maintainer has confirmed this verdict. This pull request is closed under policy clause `<clause-id>`.

Withdrawn verdict (block-comment-withdrawn): same comment edited — marker state becomes withdrawn, body replaced by:

**Triage verdict: withdrawn.**

The maintainer has withdrawn the automated verdict previously shown here. This pull request remains open and will be reviewed normally.

9. Verdict record (FR-22, NFR-12, frozen fields)

Every verdict row: verdict_id, pr_number, repo, author, disposition("below-bar"|"above-bar"|"routed"), attribution("machine"|"model"), clause_id, diff_anchor{file,start,end}, confidence(0-1|null), private_reasoning, rejected_alternatives[], machine_check_results[{check,clause_id,pass}], model_id, prompt_version, policy_version, workflow_run_id, comment_id, outcome("pending"|"confirmed"|"withdrawn"|"auto-closed"|"confirmed_by_manual_close"|null), created_at, decided_at.

Export format: JSON lines of the above (private fields included — export is maintainer-private).

10. Config file (TOML subset, frozen keys)

Path: ~/.config/corbits-triage/config.toml (CLI) / /data/config.toml (service).

[service]
port = 8787              # loopback only
data_dir = "/data"
poll_interval_s = 60

[github]
repo = "owner/name"      # GitHub.com only; reject any forge host other than github.com
app_id = ""
private_key_path = ""

[model]
provider = "stub"        # "stub" | "openai-compatible"
model_id = "stub-judge-1"
api_base = ""
api_key_path = ""

[queue]
daily_limit = 20
overflow = "queue-in-order"   # "queue-in-order" | "raise-the-bar" | "pause-new-prs"

[reconcile]
manual_close = "treat-as-confirmed"   # default; alternate "leave-unconfirmed"

[unattended]
machine_checks = false
model_judgement = false
threshold_agreement = 0.9
threshold_samples = 50

Parse with the in-repo minimal TOML parser (packages/core/src/toml.ts) — sections, string/number/bool keys, comments. No external dep.

11. Policy file (FR-2, frozen schema)

Path: policy.json committed in the adopter repo / mounted for the service.

{
  "schema_version": 1,
  "policy_version": "2026-09-13.1",
  "clauses": [
    {
      "id": "no-generated-lockfile-churn",
      "class": "machine",              // "machine" | "model" | "absolute"
      "text": "exact clause text quoted in public comments",
      "resolution": "machine:forbidden-paths",  // machine:<check> | "model" | "attestation" | "route-to-me" | "delete"
      "params": {},
      "evidence": ["repo#123"],
      "status": "active"               // "active" | "retired"
    }
  ],
  "retired": [ { "id": "old-id", "retired_at": "...", "cited_publicly": true } ]
}

Lint rules (exit 6): duplicate id; reuse of retired id (AC-3, AC-36 — any retired id, and specifically ids cited by a published comment); absolute clause whose resolution is not one of attestation/route-to-me/delete; machine clause whose check name is unknown; empty clause text.

Built-in machine checks: max-diff-lines, max-files-changed, forbidden-paths, requires-linked-issue, no-binary-files, title-pattern.

12. Judge (FR-9, FR-10, frozen)

One whole-policy call per PR. Structured JSON response required:

{ "disposition": "below-bar" | "above-bar",
  "clause_id": "…",            // required when below-bar
  "diff_anchor": {"file": "…", "start": 1, "end": 3},
  "confidence": 0.0,
  "reasoning": "private",
  "rejected": [{"clause_id": "…", "reason": "…"}] }

Prompt version constant PROMPT_VERSION = "p1". Providers: StubJudge (deterministic, fixture-driven, used by tests/calibration/injection) and OpenAICompatibleJudge (POST {api_base}/chat/completions). Validator (before any public output): schema-valid JSON; clause_id exists, active, and not machine-class; retired id → invalid; diff_anchor resolves inside the PR diff; confidence ≥ global floor. Any failure → disposition routed, apply triage: needs your read, post nothing, close nothing (FR-10). Provider error/503 → same routing, degraded: true in status, no verdict posted, nothing closed (FR-23).

13. Reconciliation (FR-19, FR-24, frozen)

Detect PR state changed directly on GitHub (close/reopen, label removed by hand, comment deleted). Never reassert stale labels or comments.

  • manual_close = "treat-as-confirmed" (default): hand-closed PR with an unconfirmed verdict → edit comment to confirmed state, remove pending label, keep PR closed, record outcome confirmed_by_manual_close (with model id, prompt version, policy version on the record) (AC-P7).
  • manual_close = "leave-unconfirmed": record the manual close (manual_close_noted), leave comment text and labels untouched, never recreate removed state.

14. Workflow runs (AC-P1)

Platform note: the corbitsdev catalog (Interchange, CorbitsCore, @corbits/artifacts) is not available in this environment (see docs/BUILD_NOTES.md). The service therefore implements the platform contract locally: every live mutation is executed inside a recorded workflow run (workflow_runs table: run_id, kind, started_at, finished_at, status, detail), kinds: intake, triage, decision, reconcile, calibrate, injection, export, diagnose. Every verdict/comment/label mutation row carries a workflow_run_id. No separate ad-hoc job queue table may drive execution. Queue figures in status are metrics over intake/triage runs and verdict rows.

15. Storage & migrations (NFR-6, NFR-10)

SQLite via bun:sqlite at <data_dir>/corbits-triage.db. Migrations are numbered, forward-only, run in a transaction; on failure roll back, keep prior schema, set settings.triage_disabled = "migration-failed"; while disabled the service serves status read-only and refuses triage with error code migration_failed (CLI exit 5). Secrets (App key, model key, admin token) are never stored in the DB — file paths only (AC-P3).

16. Calibration (FR-4, IR-10) & injection (FR-5, IR-11)

calibrate --holdback <n>: replays a held-back sample (fixtures/calibration or recorded overrides) through machine checks + stub judge. Output sections in order:

MEASURED
  agreement: …           (measured vs maintainer decisions)
  false-rejection rate: …     ← its own line, MEASURED section
  machine-checked: …
  route-to-you rate: …
  citation-validation failures: …
MODEL-ATTRIBUTED
  per-clause agreement: … (weaker diagnosis, labelled model-attributed)
GLOBAL CONFIDENCE FLOOR
  floor: …  (selected so measured false-rejection ≤ target)

Calibration writes a record keyed by judge fingerprint (model_id + prompt_version + policy_version). Changing any of the three invalidates standing and relocks model-judged unattended close (NFR-11, AC-35).

injection run: fixture cases in fixtures/injection/*.json, case classes: instruction-override, comment-smuggling, diff-smuggling, misdirected-valid-citation. Output: per-class PASS/FAIL lines then RELEASE GATE: PASS (exit 0) or RELEASE GATE: FAIL (exit 7). Any required case failing fails the gate.

17. Elicit (FR-1)

elicit --cap 2h [--corpus fixtures/corpus/sample.json]: reads past PR decisions (corpus JSON: [{pr, title, decision:"accepted"|"rejected", reason, files[]}]) via local file or GitHub API (maintainer token, GitHub.com public repos only). Interview loop: propose clause → accept/edit/skip; also accepts directly stated absolute rules and forces a resolution choice (attestation/route-to-me/delete). Every accepted clause records evidence (repo#N or maintainer-stated). Prints elapsed vs cap (elapsed 0:41 of cap 2:00) after each step; stops at cap. Non-interactive mode --yes accepts all proposals (for tests).

18. Release (NFR-8)

release/Dockerfile (oven/bun base, immutable tag corbits-triage:0.1.0, no latest), release/licence-check.ts (verifies zero runtime deps and dev-dep licence compatibility with GPLv2; fails build otherwise), release/gate.ts (runs: policy lint fixture, calibration fixture, injection suite, migration test, licence check, no-latest check; any failure → non-zero). Image and source package must contain LICENSE.md, GPL-2.0.txt, GPLv2-AI-Exception.md and THIRD_PARTY_NOTICES.md.