Skip to content

P2: web app with live progress + ignore infrastructure CI failures - #3

Merged
CodeGovindz merged 1 commit into
p1-corefrom
p2-web
Oct 26, 2026
Merged

CodeGovindz merged 1 commit into
p1-corefrom
p2-web

Conversation

@CodeGovindz

@CodeGovindz CodeGovindz commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Stacked on #2 (base p1-core). Once #2 is merged, GitHub retargets this PR to main.

What's in it

Web app

  • POST /api/verify streams the pipeline's progress events as NDJSON, ending with the full report or an error. Bad URLs get a 400, and each IP is limited to 10 runs per hour (in memory, so the limit is per server instance).
  • The home page has a PR input, three example PRs, and a live progress log with a timer. The report renders as a printed receipt: grade, trust score, problems first, evidence chips linking to GitHub, "Copy as PR comment" and "Copy share link". Opening /?pr=<url> runs that PR automatically.

Precision

  • A failed CI check isn't always a code failure. On Cap#3412, Format (Biome) died on Unexpected HTTP response: 504 before it ran Biome, and Kimi K3 used it to contradict "scoped Biome checks pass". Receipts now reads each failed run's first failure annotation. HTTP 5xx errors, runner shutdowns, "Authorization required" deploys and cancelled runs count as ignored. A contradiction that rests only on CI checks must cite one that really failed.
  • Ticked checkboxes stating checkable facts ("[x] I have added tests") are now extracted as claims.
  • Verifier reasoning is short plain English with no internal field names. The markdown report shows the self-contained claim text instead of the raw quote.
  • env.ts has turbopackIgnore hints, so the build no longer traces the whole project into the serverless function.

Verified

  • npm test: 155 passing (new tests for the rate limiter, infrastructure-failure classification and the CI-only contradiction guard)
  • npm run eval: 12/12. Cap#3412 is MIXED (67%), docs#46488 PASS, dispenso#129 MIXED.
  • npm run build and npm run lint are clean. I ran a live end-to-end call against next start (docs#46488 → PASS in 15s, events streamed) and checked the receipt in a headless browser.

Not done yet

Vercel deployment, the MCP endpoint and the Alan plugin (P3), the demo repo, and the README.

Web:
- POST /api/verify streams pipeline events as NDJSON; per-IP rate limit
  (10 runs/hour, in memory); maxDuration 300.
- Home page with PR input and examples, live progress log, and the
  report rendered as a printed receipt (problems first, evidence chips,
  copy-as-PR-comment and share link). ?pr= auto-runs a shared link.

Precision:
- Failed check runs carry their first failure annotation; HTTP 5xx,
  runner shutdowns, "Authorization required" deploys and cancelled runs
  count as ignored, not failed. Cap#3412's Format (Biome) died on an
  HTTP 504 and was producing a false CONTRADICTED.
- A CONTRADICTED verdict resting only on CI checks must cite a check
  that really failed.
- Extraction: ticked checkboxes with checkable facts are claims; claim
  text no longer starts with "After this PR". Verifier reasoning is
  short plain English without internal field names.
- Markdown report shows the self-contained claim text, not the quote.
- env.ts: turbopackIgnore hints so the bundler doesn't trace the whole
  project into the serverless function.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@CodeGovindz
CodeGovindz merged commit 50506f7 into p1-core Oct 26, 2026
CodeGovindz added a commit that referenced this pull request Oct 26, 2026
Web:
- POST /api/verify streams pipeline events as NDJSON; per-IP rate limit
  (10 runs/hour, in memory); maxDuration 300.
- Home page with PR input and examples, live progress log, and the
  report rendered as a printed receipt (problems first, evidence chips,
  copy-as-PR-comment and share link). ?pr= auto-runs a shared link.

Precision:
- Failed check runs carry their first failure annotation; HTTP 5xx,
  runner shutdowns, "Authorization required" deploys and cancelled runs
  count as ignored, not failed. Cap#3412's Format (Biome) died on an
  HTTP 504 and was producing a false CONTRADICTED.
- A CONTRADICTED verdict resting only on CI checks must cite a check
  that really failed.
- Extraction: ticked checkboxes with checkable facts are claims; claim
  text no longer starts with "After this PR". Verifier reasoning is
  short plain English without internal field names.
- Markdown report shows the self-contained claim text, not the quote.
- env.ts: turbopackIgnore hints so the bundler doesn't trace the whole
  project into the serverless function.
@CodeGovindz
CodeGovindz deleted the p2-web branch October 26, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant