Skip to content

Add merge-risk assessment for reviewed pull requests - #62

Merged
matthewp merged 2 commits into
mainfrom
merge-risk-assessment
Nov 5, 2026
Merged

matthewp merged 2 commits into
mainfrom
merge-risk-assessment

Conversation

@matthewp

@matthewp matthewp commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Changes

  • Adds a new assessment-only capability, src/merge-risk/, that posts one scored risk comment on a pull request after Factory's own Factory Review check completes. It never approves, blocks, or merges anything.
  • Scores five categories (alignment, impact, verification, rollback cost, error detection) from a bundled, non-configurable rubric (skills/merge-risk/SKILL.md, adapted from Warp's merge-risk skill) using a cheap, tool-free model call over PR metadata, the changed-file list, check conclusions, and review states — no repository checkout.
  • The overall low/medium/high score is computed deterministically in trusted code from the model's good/ok/poor category verdicts and fixed weights, not trusted to the model's own arithmetic.
  • Triggered only by check_run.completed on Factory's own review check (not on every push), so it only runs for pull requests that were actually labeled for review or had astro-reviewer requested — keeping cost down given PR volume.
  • One comment per pull request: found and updated by a marker on every re-assessment rather than accumulating new comments.
  • Wires in MergeRiskCoordinator/MergeRiskWorkflow (new Durable Object + Workflow binding, v10 migration) and a new router.ts rule that validates the check run's app id before enqueueing.

Testing

  • No new automated tests; existing suite (459 tests) still passes with no regressions, since this is additive and the router/dispatch changes are pure branches gated on a new event/check name.

Docs

  • Updated README.md with a "Merge risk" capability section, the architecture diagram, and the routing-rules summary. No external docs needed — this repo isn't published as a package.

Posts one scored comment assessing merge risk after Factory's own code review completes on a pull request. Assessment only — never approves, blocks, or merges.
Flue requires skill frontmatter metadata to be a string-to-string mapping; the nested credits object failed the build.
@matthewp
matthewp merged commit aea323d into main Nov 5, 2026
1 check passed
matthewp added a commit that referenced this pull request Nov 5, 2026
…ry (#63)

src/cloudflare.ts is the Worker's export manifest for Durable Object and Workflow classes; omitting these broke the Workers Builds deploy for #62 with "Workflow MergeRiskWorkflow must be exported" (code 10021).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant