Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .cursor-plugin/marketplace.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"name": "Receipts",
"owner": {
"name": "Govind Sharma",
"url": "https://github.com/CodeGovindz/receipts"
},
"plugins": [
{
"name": "receipts",
"source": "plugins/receipts",
"description": "Check every claim in a pull request against its real diff and CI, and check your own claims before you write them."
}
]
}
10 changes: 6 additions & 4 deletions docs/PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ Overall grade: **FAIL** if any claim is CONTRADICTED; **PASS** if all are VERIFI
├─ scripts/
│ ├─ verify.ts CLI: npx tsx scripts/verify.ts <pr-url> [--json]
│ └─ bedrock-smoke.ts calls all three models once, prints replies + latency ("npm run smoke")
├─ plugin/receipts/ Alan plugin (Cursor plugin format; see §11)
├─ .cursor-plugin/marketplace.json + plugins/receipts/ the repo is itself an Alan marketplace (see §11)
│ ├─ .cursor-plugin/plugin.json
│ ├─ mcp.json points to https://<deployment>/api/mcp
│ └─ skills/receipts-verify/SKILL.md
Expand Down Expand Up @@ -191,9 +191,9 @@ PR text is untrusted. Wrap in clear delimiters (`<pr_text>` / `<pr_data>`), tell

Format per `github.com/tryalan-ai/plugins` (README + CONTRIBUTING.md, verified):

- `plugin/receipts/.cursor-plugin/plugin.json` — fields: `name` (`receipts`), `displayName`, `version`, `description`, `author`, `homepage`, `repository`, `license: MIT`, `category`, `keywords`, `"mcpServers": "./mcp.json"`, and `"skills": "./skills/"`.
- `plugin/receipts/mcp.json` — `{ "mcpServers": { "receipts": { "type": "http", "url": "https://<deployment>/api/mcp" } } }` (remote https only; no credentials needed — server holds AWS/GitHub keys).
- `plugin/receipts/skills/receipts-verify/SKILL.md` — skill behavior: after opening/updating a PR, call `verify_pr`; on FAIL fix the code or correct the description, re-run until PASS or MIXED with UNVERIFIED items listed; attach the markdown report as a test-evidence artifact / PR comment; before writing a PR description, use `check_claims` to avoid over-claiming. `name` equals the directory; `description` states when to use it.
- `plugins/receipts/.cursor-plugin/plugin.json` — fields: `name` (`receipts`), `displayName`, `version`, `description`, `author`, `homepage`, `repository`, `license: MIT`, `category`, `keywords`, `"mcpServers": "./mcp.json"`, and `"skills": "./skills/"`.
- `plugins/receipts/mcp.json` — `{ "mcpServers": { "receipts": { "type": "http", "url": "https://<deployment>/api/mcp" } } }` (remote https only; no credentials needed — server holds AWS/GitHub keys).
- `plugins/receipts/skills/receipts-verify/SKILL.md` — skill behavior: after opening/updating a PR, call `verify_pr`; on FAIL fix the code or correct the description, re-run until PASS or MIXED with UNVERIFIED items listed; attach the markdown report as a test-evidence artifact / PR comment; before writing a PR description, use `check_claims` to avoid over-claiming. `name` equals the directory; `description` states when to use it.
- Marketplace: P3 **prepares** the PR to `tryalan-ai/plugins` (adds `plugins/receipts/`, `marketplace.json` entry, `node scripts/validate.mjs`, `node scripts/probe-auth.mjs receipts <url>` expecting `no-auth`) — **but does not open it**; the owner opens it personally.
- Validate the deployed server early with `@modelcontextprotocol/inspector`.

Expand Down Expand Up @@ -269,3 +269,5 @@ Fixtures recorded from demo PRs; vitest for `signals`/`guard`/`report` (grading
## 17. Out of scope for MVP

Posting comments from the web app, auth, database/persistence, private repos, GitHub Action (stretch goal only).

**P3 notes:** the repo root is itself an Alan plugin marketplace (`.cursor-plugin/marketplace.json` + `plugins/receipts/`), so an Alan admin can add `CodeGovindz/receipts` under Settings → Plugins → Add marketplace and install Receipts directly; the same folder can be copied into a PR to `tryalan-ai/plugins`. Alan's `scripts/validate.mjs` passes on it. The MCP route is stateless (`WebStandardStreamableHTTPServerTransport`, JSON responses) with tools `verify_pr` and `check_claims`; `check_claims` swaps the PR's text for the agent's draft claims and never writes the report cache.
22 changes: 22 additions & 0 deletions plugins/receipts/.cursor-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "receipts",
"displayName": "Receipts",
"version": "1.0.0",
"description": "Check every claim in a pull request against its real diff and CI, and check your own claims before you write them.",
"author": {
"name": "Govind Sharma"
},
"homepage": "https://github.com/CodeGovindz/receipts",
"repository": "https://github.com/CodeGovindz/receipts",
"license": "MIT",
"category": "code-and-ci",
"keywords": [
"pull-requests",
"code-review",
"verification",
"ci"
],
"logo": "assets/logo.png",
"mcpServers": "./mcp.json",
"skills": "./skills/"
}
Binary file added plugins/receipts/assets/logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
8 changes: 8 additions & 0 deletions plugins/receipts/mcp.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"mcpServers": {
"receipts": {
"type": "http",
"url": "https://DEPLOYMENT-HOST/api/mcp"
}
}
}
40 changes: 40 additions & 0 deletions plugins/receipts/skills/receipts-verify/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
name: receipts-verify
description: Check that what a pull request says it does matches its real diff and CI. Use after opening or updating a pull request, before writing or editing a PR description, and when reviewing someone else's (or another agent's) PR that claims "added tests", "all tests pass", "no breaking changes" or similar.
---

# Receipts: every claim, with a receipt

The Receipts MCP server checks claims about a GitHub pull request against the PR's actual diff, changed test files and CI checks. Hosts prefix tool names (in Alan: `receipts_receipts__verify_pr` and `receipts_receipts__check_claims`); the names below are the upstream ones.

Each claim comes back as:

- **VERIFIED**: the diff or CI supports it, with links to the lines or checks.
- **CONTRADICTED**: the evidence shows it is false, with links.
- **UNVERIFIED**: nothing checkable supports or refutes it (for example "tests pass locally", or CI skipped the test jobs).

The PR gets **PASS** (all verified), **FAIL** (anything contradicted) or **MIXED**.

## Before you write a PR description

1. Push the branch and open the PR (a short placeholder description is fine).
2. Call `check_claims` with the `pr_url` and the statements you plan to make, one per item, for example `["Adds tests for the login timeout", "All tests pass in CI", "No public API changes"]`.
3. Drop or rewrite every CONTRADICTED claim. For each UNVERIFIED one, either make it checkable (add the test, wait for CI) or say plainly in the description that it is unverified and why.
4. Write the final description from what survived.

## After you open or update a PR

1. Call `verify_pr` with the `pr_url`. Pass `fresh: true` after pushing new commits.
2. If the grade is **FAIL**, fix the code or correct the description, push, and run `verify_pr` again with `fresh: true`.
3. Stop when the grade is PASS, or MIXED with every UNVERIFIED item explained in the description.
4. Post the markdown from the first content block as a PR comment with your GitHub tools, and attach it to the task as testing evidence. Its first line is the marker `<!-- receipts:report -->`; if an earlier Receipts comment exists, update it instead of adding another.

## Reviewing someone else's PR

Run `verify_pr` and lead your review with the CONTRADICTED items, then the UNVERIFIED ones. Quote the evidence links instead of re-deriving them.

## Rules

- Never claim a check passed because Receipts said UNVERIFIED. Unverified means nobody has shown it yet.
- A run takes 10 to 60 seconds. Don't call it in a loop without pushing changes in between.
- Only public GitHub repositories are supported. If the tool errors (rate limit, private repo), say so instead of guessing the verdicts.
34 changes: 34 additions & 0 deletions src/app/api/mcp/route.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
/**
* Remote MCP server (Streamable HTTP, stateless): a fresh server and
* transport per request, so it works on serverless functions. JSON responses
* instead of SSE, since every tool call finishes in one response.
*/
import { WebStandardStreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/webStandardStreamableHttp.js";
import { clientKey } from "@/lib/ratelimit";
import { createReceiptsMcpServer } from "@/mcp/server";

export const maxDuration = 300;

export async function POST(request: Request): Promise<Response> {
const server = createReceiptsMcpServer(clientKey(request.headers));
const transport = new WebStandardStreamableHTTPServerTransport({
sessionIdGenerator: undefined,
enableJsonResponse: true,
});
await server.connect(transport);
try {
return await transport.handleRequest(request);
} finally {
await server.close();
}
}

function methodNotAllowed(): Response {
return Response.json(
{ jsonrpc: "2.0", error: { code: -32000, message: "Method not allowed: this MCP server is stateless, use POST." }, id: null },
{ status: 405, headers: { Allow: "POST" } },
);
}

export const GET = methodNotAllowed;
export const DELETE = methodNotAllowed;
Binary file removed src/app/favicon.ico
Binary file not shown.
Binary file added src/app/icon.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
45 changes: 44 additions & 1 deletion src/core/pipeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ export interface RunOnBundleOptions {
config: ReceiptsConfig;
onEvent?: (event: PipelineEvent) => void;
fresh?: boolean;
/** Store the report in the cache (default true; false for draft checks, whose bundle isn't the real PR). */
cache?: boolean;
/** Time the GitHub fetch took, for honest timings on live runs (default 0). */
fetchMs?: number;
}
Expand Down Expand Up @@ -152,7 +154,7 @@ export async function runReceiptsOnBundle(
verifyAttempts: verification.attempts,
},
});
reportCache.set(key, report);
if (options.cache !== false) reportCache.set(key, report);
emit({ type: "done", report });
return report;
} catch (error) {
Expand Down Expand Up @@ -196,3 +198,44 @@ export async function runReceipts(prUrl: string, options: RunReceiptsOptions = {
throw error;
}
}

/** Max draft claims check_claims accepts in one call. */
export const MAX_DRAFT_CLAIMS = 12;

/**
* Checks claims an agent is ABOUT to make (e.g. a draft PR description)
* against the PR's real diff and CI, before it writes them. The drafts
* replace the PR's own title, body and commits as the extraction source, so
* the quote guard validates against exactly what the agent submitted.
* Never cached: the same head SHA answers different drafts.
*/
export async function runClaimCheck(
prUrl: string,
claims: string[],
options: Omit<RunReceiptsOptions, "fresh" | "onBundle"> = {},
): Promise<Report> {
loadDotEnv();
const config = options.config ?? getConfig();
const drafts = claims.map((claim) => claim.trim()).filter((claim) => claim !== "");
if (drafts.length === 0) throw new Error("Provide at least one claim to check.");
if (drafts.length > MAX_DRAFT_CLAIMS) {
throw new Error(`Check at most ${MAX_DRAFT_CLAIMS} claims per call (got ${drafts.length}).`);
}

const fetchStarted = Date.now();
options.onEvent?.({ type: "fetching" });
const bundle = await fetchPrBundle(prUrl, { githubToken: config.githubToken });
const draftBundle: PrBundle = {
...bundle,
title: "",
body: drafts.map((claim) => `- ${claim}`).join("\n"),
commits: [],
};
return runReceiptsOnBundle(draftBundle, {
config,
onEvent: options.onEvent,
fresh: true,
cache: false,
fetchMs: Date.now() - fetchStarted,
});
}
124 changes: 124 additions & 0 deletions src/mcp/server.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
/**
* Receipts as an MCP server (plan §10/§11). Two read-only tools:
* - verify_pr: check every claim in an existing PR.
* - check_claims: check claims an agent is about to write, before it writes them.
* Each result carries the GitHub-comment-ready markdown plus a compact JSON
* summary. The server has no GitHub write access: agents post the markdown
* with their own GitHub tools.
*/
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { z } from "zod";
import { MAX_DRAFT_CLAIMS, runClaimCheck, runReceipts } from "@/core/pipeline";
import { toMarkdown } from "@/core/report";
import type { Report } from "@/core/types";
import { takeRun } from "@/lib/ratelimit";

type ToolResult = {
content: Array<{ type: "text"; text: string }>;
isError?: boolean;
};

/** The fields an agent acts on, without the full report's timings and quotes. */
function summarize(report: Report) {
return {
prUrl: report.prUrl,
headSha: report.headSha,
grade: report.grade,
trustScore: report.trustScore,
claims: report.claims.map((result) => ({
claim: result.claim.text,
verdict: result.verdict,
reasoning: result.reasoning,
evidence: result.evidence.map((item) => item.url).filter((url): url is string => url !== null),
})),
signals: report.signals,
};
}

function success(report: Report): ToolResult {
return {
content: [
{ type: "text", text: toMarkdown(report) },
{ type: "text", text: JSON.stringify(summarize(report)) },
],
};
}

function failure(error: unknown): ToolResult {
return {
content: [{ type: "text", text: `Receipts failed: ${error instanceof Error ? error.message : String(error)}` }],
isError: true,
};
}

/** One server per request (stateless transport); `clientKey` rate-limits tool calls. */
export function createReceiptsMcpServer(clientKey: string): McpServer {
const server = new McpServer({ name: "receipts", version: "1.0.0" });

const limited = (): ToolResult | null => {
const limit = takeRun(clientKey);
if (limit.ok) return null;
return failure(
new Error(`rate limit reached (10 runs per hour); retry in ${Math.ceil(limit.retryAfterSec / 60)} min.`),
);
};

server.registerTool(
"verify_pr",
{
title: "Verify a pull request's claims",
description:
"Extracts every checkable claim from a GitHub pull request's title, description and commits " +
"(\"added tests\", \"all tests pass\", \"no breaking changes\"…) and checks each one against the " +
"real diff and CI. Each claim gets VERIFIED, CONTRADICTED or UNVERIFIED with evidence links; the " +
"PR gets PASS (all verified), FAIL (any contradicted) or MIXED. Takes 10-60 s. Returns a " +
"markdown report ready to post as a PR comment, then a JSON summary.",
inputSchema: {
pr_url: z.string().describe("Public GitHub PR URL, e.g. https://github.com/owner/repo/pull/123"),
fresh: z.boolean().optional().describe("Skip the cache and re-check (e.g. after pushing fixes)."),
},
annotations: { readOnlyHint: true, openWorldHint: true },
},
async ({ pr_url, fresh }) => {
const blocked = limited();
if (blocked) return blocked;
try {
return success(await runReceipts(pr_url, { fresh }));
} catch (error) {
return failure(error);
}
},
);

server.registerTool(
"check_claims",
{
title: "Check claims before writing them",
description:
"Checks claims you are ABOUT to make about a pull request (for example the bullet points of a " +
"PR description you are drafting) against its real diff and CI, before you post them. Use it " +
"to avoid over-claiming: rewrite or drop anything CONTRADICTED, and say plainly what stays " +
"UNVERIFIED. The PR must already exist (push the branch and open it first).",
inputSchema: {
pr_url: z.string().describe("Public GitHub PR URL the claims are about."),
claims: z
.array(z.string())
.min(1)
.max(MAX_DRAFT_CLAIMS)
.describe(`1-${MAX_DRAFT_CLAIMS} claims, one statement each, e.g. "Adds tests for the login timeout".`),
},
annotations: { readOnlyHint: true, openWorldHint: true },
},
async ({ pr_url, claims }) => {
const blocked = limited();
if (blocked) return blocked;
try {
return success(await runClaimCheck(pr_url, claims));
} catch (error) {
return failure(error);
}
},
);

return server;
}