Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/637-revision-budget.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
A review that requests changes now starts a fix through a budget instead of without one: a third-party review bot can cause at most one fix per pull request, and the platform's own reviewer at most three per head (a repository can lower its own ceiling with the `repo:<id>:max_revisions` setting). A person's requests are never counted, a repeated request is still answered by the existing duplicate-fingerprint guard before any budget is spent, and every refusal writes one `trigger.revision_refused` audit row. The count resets when the pull request head moves, and a request that arrives while a review is already running keeps its author into the replay, so the fix it starts is counted exactly like one that started immediately.
19 changes: 19 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -1693,6 +1693,25 @@ and starts nothing. Every accepted trigger writes `trigger.mention` with the
comment id, the head SHA, the actor and the outcome (`started`, `recorded`,
`parked`), and answers in the thread.

A review that requests changes may start a fix, and that path is budgeted —
`src/lib/revisionBudget.ts` holds the rule. The author of the review decides the
bound: a person is never counted, a third-party review bot may cause one fix per
pull request whichever head it reviewed, and the platform's own reviewer (the
installation's `<app slug>[bot]` account) may cause `max_revisions` at one head —
three by default, overridable per repository with the setting
`repo:<id>:max_revisions`. The existing fingerprint dedupe answers before the
budget, so a repeated request never spends it; the count follows the fix that
actually starts, so a request parked behind a running job counts when it is
replayed rather than when it parks, and several updates merged into one job
count once. The parked row carries the author at its top level, and the review
job's own follow-ups park as the platform reviewer, so a replayed request is
decided exactly like one that starts on arrival. When several requests merge
into one row, a person's request wins — a merged fix is never refused by a bot's
budget — and otherwise the strictest bot class present is charged, so a
third-party bot's single fix is not hidden behind the platform's larger
allowance. A refusal writes one `trigger.revision_refused` row carrying the
actor class, the head and the reason, and no job.

The inbox is also available as `GET /api/inbox/events`, a per-user SSE stream.
The server reuses the cached inbox snapshot, sends the server-rendered inbox HTML
when the question signature changes, and the browser replaces the section and
Expand Down
13 changes: 11 additions & 2 deletions src/app.ts
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ import { isWorkItemState, MAINTAINER_ONLY_STATES, WORK_ITEM_TRANSITIONS, type Wo
import { parseStageMode } from './lib/stages.js'
import { issuePriority, issueState, shouldTriage } from './lib/issueIntake.js'
import { parseIssueCommand, parseMentionTrigger, type IssueCommandInput, type MentionTriggerInput } from './lib/issueCommands.js'
import { classifyReviewAuthor, type ReviewAuthor } from './lib/revisionBudget.js'
import { untrustedAuditDetail, untrustedIssueGoal, untrustedMeta, type UntrustedText } from './lib/untrusted.js'
import { scanUntrusted, UNTRUSTED_EVENT_KIND } from './lib/firewall.js'
import { VERSION } from './version.js'
Expand Down Expand Up @@ -5894,6 +5895,8 @@ export async function buildApp(options: BuildOptions): Promise<BuiltApp> {
// the agent was told "review requested changes" and nothing else.
let instruction = ''
let reconcile = false
// #637: who reviewed decides whether this fix is counted against a budget.
let reviewAuthor: ReviewAuthor | undefined
if (event === 'check_run') {
const check = typeof body.check_run === 'object' && body.check_run !== null ? (body.check_run as Record<string, unknown>) : {}
const conclusion = String(check.conclusion ?? '')
Expand All @@ -5914,6 +5917,12 @@ export async function buildApp(options: BuildOptions): Promise<BuiltApp> {
reconcile = review.state === 'approved'
reason = review.state === 'approved' ? 'review approved' : 'review requested changes'
instruction = review.state === 'changes_requested' ? normalizeReviewInstruction(review.body) : ''
const reviewer = typeof review.user === 'object' && review.user !== null ? (review.user as Record<string, unknown>) : {}
reviewAuthor = classifyReviewAuthor(
typeof reviewer.login === 'string' ? reviewer.login : '',
typeof reviewer.type === 'string' ? reviewer.type : '',
`${config.githubApp.slug}[bot]`,
)
const pull = typeof body.pull_request === 'object' && body.pull_request !== null ? (body.pull_request as typeof pulls[number]) : undefined
if (pull !== undefined) pulls.push(pull)
} else {
Expand All @@ -5933,8 +5942,8 @@ export async function buildApp(options: BuildOptions): Promise<BuiltApp> {
log.info('github merge reconcile', { repo: `${repo.owner}/${repo.name}`, pr: number, reason })
continue
}
const result = await jobs.followUp(repo, { prUrl, branch, baseBranch, reason, headSha, ...(instruction === '' ? {} : { instruction }) })
log.info('github follow-up', { repo: `${repo.owner}/${repo.name}`, pr: number, ok: result.ok, code: result.code, trace: result.job?.id })
const result = await jobs.followUp(repo, { prUrl, branch, baseBranch, reason, headSha, ...(instruction === '' ? {} : { instruction }), ...(reviewAuthor === undefined ? {} : { reviewAuthor }) })
log.info('github follow-up', { repo: `${repo.owner}/${repo.name}`, pr: number, ok: result.ok, code: result.code, author: reviewAuthor, trace: result.job?.id })
}
}

Expand Down
85 changes: 81 additions & 4 deletions src/jobs/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ import { evaluateTestEvidence, type VerificationResult } from '../lib/verificati
import { observedAttribution } from '../lib/teamStats.js'
import { reviewInstructionLine } from '../lib/followUpInstruction.js'
import { decideMention, mentionIdentity, mentionJobKind, type MentionTarget } from '../lib/mentionGovernance.js'
import { DEFAULT_MAX_REVISIONS, decideRevision, maxRevisionsSettingKey, parseMaxRevisions, revisionCounterKey, type ReviewAuthor } from '../lib/revisionBudget.js'
import { sweepExpiredTrials, type TrialSweepResult } from '../users/trial.js'
import { decryptSecret, encryptSecret, randomId, randomToken, safeEqual, sha256Hex } from '../lib/secrets.js'
import { addMinutes, nowIso } from '../lib/time.js'
Expand Down Expand Up @@ -126,7 +127,9 @@ function scheduleAnchor(row: ScheduleRow, repo: RepoRow): number {
// A follow-up is refused by its own duplicate key before it reaches `enqueue`, so
// its result is the enqueue shape plus that one local code. The command-level
// codes stay out of this union: an enqueue refusal keeps its own code and detail.
export type FollowUpResult = Omit<EnqueueResult, 'code'> & { code?: EnqueueRefusalCode | 'DUPLICATE_FOLLOWUP' }
// #637: `REVISION_BUDGET` is the second local refusal: a review-triggered fix
// that the review budget will not admit, decided before `enqueue`.
export type FollowUpResult = Omit<EnqueueResult, 'code'> & { code?: EnqueueRefusalCode | 'DUPLICATE_FOLLOWUP' | 'REVISION_BUDGET' }

// #283: the whole platform stop is one `settings` row, and every reader goes back
// to the row instead of to a field on this object. A cached copy is exactly how a
Expand Down Expand Up @@ -230,6 +233,12 @@ export interface FollowUpRequest {
headSha: string
instruction?: string
trigger?: FollowUpTrigger
/**
* #637: who asked for this fix. Absent means the request did not come from a
* review (a check failure, a manual retry), which is not counted against any
* review budget.
*/
reviewAuthor?: ReviewAuthor
}

// #637: one mention, as the webhook read it. The target is what decides the kind
Expand All @@ -253,6 +262,7 @@ interface ParkedFollowUpRequest {
reason: string
instruction?: string
trigger?: FollowUpTrigger
reviewAuthor?: ReviewAuthor
}

// One pull request at one head has ONE parked request, whatever arrived while the
Expand All @@ -275,15 +285,31 @@ function pendingFollowUpKey(repoId: string, prUrl: string, headSha: string): str
return `pending_followup:${repoId}:${prUrl}:${headSha}`.slice(0, 300)
}

// #637: one parked row may hold several requests, but the replay starts one job,
// so the row carries one author. A person's request wins: merging must never let
// a budget refuse a job the budget would have allowed on its own. Otherwise the
// strictest bot class present wins, so a third-party bot's single fix is charged
// to it rather than hidden behind the platform's own larger allowance.
function parkedReviewAuthor(requests: ParkedFollowUpRequest[]): ReviewAuthor | undefined {
const authors = new Set<ReviewAuthor>()
for (const request of requests) {
if (request.reviewAuthor !== undefined) authors.add(request.reviewAuthor)
}
if (authors.has('human')) return 'human'
if (authors.has('bot')) return 'bot'
if (authors.has('reviewer')) return 'reviewer'
return undefined
}

function mergeParkedFollowUp(existingJson: string | undefined, input: FollowUpRequest): string {
const requests: ParkedFollowUpRequest[] = []
if (existingJson !== undefined) {
try {
const parsed = JSON.parse(existingJson) as Partial<ParkedFollowUpRequest> & { requests?: ParkedFollowUpRequest[] }
if (Array.isArray(parsed.requests)) {
for (const request of parsed.requests) requests.push({ reason: request.reason, ...(request.instruction === undefined ? {} : { instruction: request.instruction }), ...(request.trigger === undefined ? {} : { trigger: request.trigger }) })
for (const request of parsed.requests) requests.push({ reason: request.reason, ...(request.instruction === undefined ? {} : { instruction: request.instruction }), ...(request.trigger === undefined ? {} : { trigger: request.trigger }), ...(request.reviewAuthor === undefined ? {} : { reviewAuthor: request.reviewAuthor }) })
} else if (typeof parsed.reason === 'string') {
requests.push({ reason: parsed.reason, ...(parsed.instruction === undefined ? {} : { instruction: parsed.instruction }), ...(parsed.trigger === undefined ? {} : { trigger: parsed.trigger }) })
requests.push({ reason: parsed.reason, ...(parsed.instruction === undefined ? {} : { instruction: parsed.instruction }), ...(parsed.trigger === undefined ? {} : { trigger: parsed.trigger }), ...(parsed.reviewAuthor === undefined ? {} : { reviewAuthor: parsed.reviewAuthor }) })
}
} catch {
// A corrupt parked row is replaced by the request in hand: losing the
Expand All @@ -294,6 +320,7 @@ function mergeParkedFollowUp(existingJson: string | undefined, input: FollowUpRe
reason: input.reason,
...(input.instruction === undefined ? {} : { instruction: input.instruction }),
...(input.trigger === undefined ? {} : { trigger: input.trigger }),
...(input.reviewAuthor === undefined ? {} : { reviewAuthor: input.reviewAuthor }),
}
const duplicate = requests.some((request) =>
request.reason === incoming.reason &&
Expand All @@ -302,6 +329,7 @@ function mergeParkedFollowUp(existingJson: string | undefined, input: FollowUpRe
if (!duplicate) requests.push(incoming)
const reasons = [...new Set(requests.map((request) => request.reason))]
const instructions = requests.flatMap((request) => (request.instruction === undefined || request.instruction === '' ? [] : [request.instruction]))
const author = parkedReviewAuthor(requests)
const merged = {
prUrl: input.prUrl,
branch: input.branch,
Expand All @@ -310,6 +338,10 @@ function mergeParkedFollowUp(existingJson: string | undefined, input: FollowUpRe
headSha: input.headSha,
...(instructions.length === 0 ? {} : { instruction: instructions.join(' — also: ') }),
...(input.trigger === undefined ? {} : { trigger: input.trigger }),
// #637: the replay reads this row as a `FollowUpRequest`, so the author has
// to be at the top level; keeping it only inside `requests` made every
// parked fix start with no budget decision and no counter.
...(author === undefined ? {} : { reviewAuthor: author }),
requests,
}
return JSON.stringify(merged)
Expand Down Expand Up @@ -2766,7 +2798,12 @@ export class Jobs {
this.store.deleteSetting(row.key)
let input: FollowUpRequest
try {
input = JSON.parse(row.value) as typeof input
const parsed = JSON.parse(row.value) as FollowUpRequest & { requests?: ParkedFollowUpRequest[] }
// #637: a row parked by the previous release carried the author only per
// request, and the replay must not lose it: without this a request parked
// before the upgrade starts its fix with no budget decision at all.
const author = parsed.reviewAuthor ?? (Array.isArray(parsed.requests) ? parkedReviewAuthor(parsed.requests) : undefined)
input = author === undefined ? parsed : { ...parsed, reviewAuthor: author }
} catch {
continue
}
Expand Down Expand Up @@ -2983,6 +3020,21 @@ export class Jobs {
const fingerprint = followUpFingerprint(input.reason, input.instruction)
const key = `followup:${repo.id}:${fingerprint}:${input.prUrl}:${input.headSha}`.slice(0, 300)
if (this.store.getSetting(key) !== undefined) return { ok: false, error: '这个失败已经跟进过', code: 'DUPLICATE_FOLLOWUP' }
// #637: the review budget answers after the fingerprint, so a repeated
// request never spends it. A person is never refused here.
if (input.reviewAuthor !== undefined) {
const budget = this.revisionBudget(repo, input, input.reviewAuthor)
if (!budget.allow) {
this.store.audit(repo.user_id, 'trigger.revision_refused', `${input.prUrl} ${input.headSha} author=${input.reviewAuthor} used=${budget.used} reason=${budget.reason}`)
return {
ok: false,
error: budget.reason === 'bot-once'
? '另一个机器人的评审在这个 PR 上已经触发过一次修复。'
: `这个 head 的自动评审修复已达到上限(${budget.used})。`,
code: 'REVISION_BUDGET',
}
}
}
const activeJob = this.store.activeJobForRepo(repo.id)
if (activeJob !== undefined) {
if (activeJob.kind === 'review') {
Expand All @@ -3005,9 +3057,29 @@ export class Jobs {
].join('\n')
const result = await this.enqueue(repo, 'followup', goal, 'iterate', { branch: input.branch, prUrl: input.prUrl, baseBranch: input.baseBranch })
if (!result.ok) this.store.deleteSetting(key)
// The count follows the fix that actually started. A parked request is
// counted when it is replayed through this method, not when it parks, so a
// merge of several updates into one job is one fix and one count.
if (result.ok && input.reviewAuthor !== undefined) this.recordRevision(repo, input, input.reviewAuthor)
return result
}

// #637: the review budget is read from one place, the settings table, so it
// survives a restart and a parked request is counted when it actually starts.
private revisionBudget(repo: RepoRow, input: FollowUpRequest, author: ReviewAuthor): ReturnType<typeof decideRevision> {
if (author === 'human') return decideRevision({ author, used: 0, limit: DEFAULT_MAX_REVISIONS })
const used = Number(this.store.getSetting(revisionCounterKey(author, repo.id, input.prUrl, input.headSha)) ?? '0')
const limit = parseMaxRevisions(this.store.getSetting(maxRevisionsSettingKey(repo.id)))
return decideRevision({ author, used: Number.isInteger(used) ? used : 0, limit })
}

private recordRevision(repo: RepoRow, input: FollowUpRequest, author: ReviewAuthor): void {
if (author === 'human') return
const key = revisionCounterKey(author, repo.id, input.prUrl, input.headSha)
const used = Number(this.store.getSetting(key) ?? '0')
this.store.setSetting(key, String((Number.isInteger(used) ? used : 0) + 1))
}

// #637: a mention is the third way a job starts, beside the scheduler and the
// command surface. The rule lives in `lib/mentionGovernance.ts`; this method is
// only what the rule needs read from the store and what it writes back.
Expand Down Expand Up @@ -3774,6 +3846,11 @@ export class Jobs {
// not a type error, it is a silent loss for every follow-up parked behind
// a review job.
...(follow.instruction === undefined ? {} : { instruction: follow.instruction }),
// #637: these follow-ups are the platform reviewer's own request — the
// review job posts as the installation's `<slug>[bot]` account — so they
// spend the reviewer budget like the webhook's request would. Without
// this the normal review path never reached `followUp` with an author.
reviewAuthor: 'reviewer' as const,
}
// #637: a follow-up the review itself asked for parks in the same slot as
// an update that arrived mid-review, keyed by pull request and head, so the
Expand Down
69 changes: 69 additions & 0 deletions src/lib/revisionBudget.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
// #637: a review that requests changes may start a fix, and a loop of such
// reviews must not spend model budget without a bound. The number of fixes a
// review may cause depends on who reviewed: a person is not counted, a
// third-party review bot may cause one fix per pull request, and the platform's
// own reviewer may cause `max_revisions` at one head. The rule is a pure
// function so every combination is cheaper to prove here than through a webhook
// and a database.

export type ReviewAuthor = 'human' | 'bot' | 'reviewer'

/** Default ceiling for the platform reviewer's own loop; a repository may lower it. */
export const DEFAULT_MAX_REVISIONS = 3

/** Where a repository overrides `DEFAULT_MAX_REVISIONS`. */
export function maxRevisionsSettingKey(repoId: string): string {
return `repo:${repoId}:max_revisions`
}

export interface RevisionBudgetFacts {
author: ReviewAuthor
/** Fixes already started for this identity. */
used: number
/** The repository's ceiling for the platform reviewer. */
limit: number
}

export type RevisionBudgetDecision =
| { allow: true; counted: boolean; used: number }
| { allow: false; reason: 'bot-once' | 'reviewer-limit'; used: number }

export function decideRevision(facts: RevisionBudgetFacts): RevisionBudgetDecision {
// A person asking for a change is never met with a budget refusal: the
// maintainer's own words are the request, and the fingerprint dedupe below is
// what keeps a repeated request from starting twice.
if (facts.author === 'human') return { allow: true, counted: false, used: facts.used }
if (facts.author === 'bot') {
// Another bot reviewing this product must not be able to drive a loop: one
// fix per pull request, whichever head it reviewed and however it worded it.
return facts.used >= 1
? { allow: false, reason: 'bot-once', used: facts.used }
: { allow: true, counted: true, used: 1 }
}
const limit = Number.isInteger(facts.limit) && facts.limit > 0 ? facts.limit : DEFAULT_MAX_REVISIONS
return facts.used >= limit
? { allow: false, reason: 'reviewer-limit', used: facts.used }
: { allow: true, counted: true, used: facts.used + 1 }
}

export function parseMaxRevisions(value: string | undefined): number {
const parsed = Number(value ?? '')
return Number.isInteger(parsed) && parsed > 0 ? parsed : DEFAULT_MAX_REVISIONS
}

/** The counter for one identity: a bot counts per pull request, the reviewer per head. */
export function revisionCounterKey(author: 'bot' | 'reviewer', repoId: string, prUrl: string, headSha: string): string {
return author === 'bot' ? `revision:bot:${repoId}:${prUrl}` : `revision:reviewer:${repoId}:${headSha}:${prUrl}`
}

/**
* The platform reviewer is the GitHub App's bot account (`<slug>[bot]`), because
* that is the identity an installation token posts as. Any other bot account is
* a third party; everything else is a person. The login check comes first
* because an App bot would otherwise be classified as a third party.
*/
export function classifyReviewAuthor(login: string, type: string, reviewerLogin: string): ReviewAuthor {
if (login !== '' && reviewerLogin !== '' && login.toLowerCase() === reviewerLogin.toLowerCase()) return 'reviewer'
if (type.toLowerCase() === 'bot' || login.toLowerCase().endsWith('[bot]')) return 'bot'
return 'human'
}
Loading
Loading