Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/366-temporal-delta.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
An iterate job's goal now carries one line of temporal context — `Since 4f2c1a9: 3 commits ahead, 2 merged PRs (#812, #815)` — built from the previous finished run's base SHA, and the job page shows the same line. A first run says `Since: no previous run`, and a compare that failed or a page walk that hit its cap is marked `partial` instead of reporting a smaller number.
28 changes: 25 additions & 3 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -673,8 +673,8 @@ recorded". `scripts/restore-drill.mjs --status-file` is the writer.
terminal state (`merged` or `closed`) is left exactly as it is: the expiry is
still recorded in the audit row and in the `plan_expired:<job>` settings key,
but no "已过期关闭" notice is sent for an item the pass did not end (#408).
Iterate jobs preflight the target branch, record the base SHA
and warn when the fork is behind. An open work item changes what the next
Iterate jobs preflight the target branch, record the base SHA,
warn when the fork is behind, and add the temporal delta line described below. An open work item changes what the next
scheduled run continues; it never blocks a due repository (#219). Work-item
states do not gate the scheduler; see "Scheduled work, continuity and work
items" below.
Expand Down Expand Up @@ -1326,7 +1326,29 @@ request — so a closed issue or a merged pull request is dropped from the block
and counted instead of being carried into the next run. The block states what it
did in its header, `carry-over: 1 used, 1 dropped (closed/merged)`, with an
`, N unverified` segment when a read failed; a failed read keeps its line rather
than dropping it silently, and the job page shows the same header line. Each finished run is classified as `pr`, `issue`, `review`, `zero`,
than dropping it silently, and the job page shows the same header line.

The goal also carries one **temporal delta** line (#366), built by
`summariseDelta()` in `src/lib/temporal.ts` from two reads plus the commit
lookup that dates them, all against the previous finished run's base:
`compareCommits()`, `getCommitMeta()` for that base's `committedAt`, and then
`listMergedPullRequestsSince()` for that window. The base comes from
`store.latestFinishedBaseSha()`, one indexed question rather than a page of the
newest jobs, so a repository whose newest rows are all unfinished is not read as
a first run. The line names the short
previous base, the commit count and the merged pull requests, for example
`Since 4f2c1a9: 3 commits ahead, 2 merged PRs (#812, #815)`; a first run says
`Since: no previous run` and makes neither read. A failed compare prints
`delta unavailable (partial)` and a capped page walk prints `merged PRs partial`
— never the smaller number that happened to fit — because a delta that
understates the change is worse than one that admits it is incomplete. The
result is stored in `jobs.temporal_json` (`sinceSha`, `status`, `aheadBy`,
`mergedPrs`, `complete`), the additive column that lets a later slice compare two
deltas, and the job page shows the same line — and renders nothing for a row
whose fields are not the shape the summariser reads. The line is placed before
the 4000-character cap is applied to the goal body, so a long goal cannot squeeze
it out. Nothing reads the delta to make a decision yet; slice 1 only records and
displays it. Each finished run is classified as `pr`, `issue`, `review`, `zero`,
`meta` or `failed`. A meta issue
(`存量/盘点/清账/reconcile/inventory/backlog audit`) is closed on the tracking
board and notifies the owner instead of counting as a productive run. Two
Expand Down
18 changes: 17 additions & 1 deletion src/db.ts
Original file line number Diff line number Diff line change
Expand Up @@ -427,6 +427,9 @@ export interface JobRow {
plan_allow_paths: string | null
plan_deny_paths: string | null
base_sha: string | null
// #366: one run's delta against the previous run's base, as JSON. Additive;
// NULL for every run that predates it and for a first run on a repository.
temporal_json: string | null
callback_fingerprint: string | null
path_guard_json: string | null
verification_json: string | null
Expand Down Expand Up @@ -1267,6 +1270,7 @@ CREATE TABLE IF NOT EXISTS jobs (
plan_allow_paths TEXT,
plan_deny_paths TEXT,
base_sha TEXT,
temporal_json TEXT,
callback_fingerprint TEXT,
path_guard_json TEXT,
verification_json TEXT,
Expand Down Expand Up @@ -2631,6 +2635,7 @@ export class Store {
if (!jobColumns.has('plan_allow_paths')) this.run('ALTER TABLE jobs ADD COLUMN plan_allow_paths TEXT')
if (!jobColumns.has('plan_deny_paths')) this.run('ALTER TABLE jobs ADD COLUMN plan_deny_paths TEXT')
if (!jobColumns.has('base_sha')) this.run('ALTER TABLE jobs ADD COLUMN base_sha TEXT')
if (!jobColumns.has('temporal_json')) this.run('ALTER TABLE jobs ADD COLUMN temporal_json TEXT')
if (!jobColumns.has('callback_fingerprint')) this.run('ALTER TABLE jobs ADD COLUMN callback_fingerprint TEXT')
if (!jobColumns.has('path_guard_json')) this.run('ALTER TABLE jobs ADD COLUMN path_guard_json TEXT')
if (!jobColumns.has('verification_json')) this.run('ALTER TABLE jobs ADD COLUMN verification_json TEXT')
Expand Down Expand Up @@ -5615,6 +5620,17 @@ export class Store {
return this.all<JobRow>('SELECT * FROM jobs WHERE repo_id = ? ORDER BY created_at DESC, rowid DESC LIMIT ?', repoId, limit)
}

// #366: the base SHA of the newest finished job that recorded one. A window
// over `listJobsForRepo` would answer "no previous run" on a busy repository
// whose newest twenty rows are unfinished, so the question is asked directly.
latestFinishedBaseSha(repoId: string): string | null {
const row = this.get<{ base_sha: string | null }>(
"SELECT base_sha FROM jobs WHERE repo_id = ? AND finished_at IS NOT NULL AND base_sha IS NOT NULL AND base_sha != '' ORDER BY finished_at DESC, rowid DESC LIMIT 1",
repoId,
)
return row?.base_sha ?? null
}

// #360 review: a work item's job is looked up by the item, not through a window
// of the repository's newest jobs. The window said "no job is attached" for a
// work item whose job was older than the limit — a false statement, and a
Expand Down Expand Up @@ -5833,7 +5849,7 @@ export class Store {
}


updateJob(id: string, patch: Partial<Pick<JobRow, 'status' | 'branch' | 'pr_url' | 'issue_url' | 'pr_state' | 'review_score' | 'attribution_json' | 'evidence_json' | 'checkpoint' | 'team_stats_json' | 'summary' | 'error' | 'error_code' | 'cost_usd' | 'started_at' | 'last_callback_at' | 'finished_at' | 'base_branch' | 'work_item_id' | 'priority' | 'plan_approved_at' | 'plan_approved_by' | 'plan_rejected_at' | 'plan_allow_paths' | 'plan_deny_paths' | 'base_sha' | 'path_guard_json' | 'verification_json' | 'continuity_json' | 'model_json' | 'provider_model' | 'provider_region' | 'provider_label' | 'provider_id' | 'output_kind' | 'work_type' | 'triage_json' | 'continuity_used' | 'retry_attempt' | 'retry_of' | 'lease_owner' | 'lease_until' | 'callback_secret_enc' | 'platform_token_enc'>>): void {
updateJob(id: string, patch: Partial<Pick<JobRow, 'status' | 'branch' | 'pr_url' | 'issue_url' | 'pr_state' | 'review_score' | 'attribution_json' | 'evidence_json' | 'checkpoint' | 'team_stats_json' | 'summary' | 'error' | 'error_code' | 'cost_usd' | 'started_at' | 'last_callback_at' | 'finished_at' | 'base_branch' | 'work_item_id' | 'priority' | 'plan_approved_at' | 'plan_approved_by' | 'plan_rejected_at' | 'plan_allow_paths' | 'plan_deny_paths' | 'base_sha' | 'temporal_json' | 'path_guard_json' | 'verification_json' | 'continuity_json' | 'model_json' | 'provider_model' | 'provider_region' | 'provider_label' | 'provider_id' | 'output_kind' | 'work_type' | 'triage_json' | 'continuity_used' | 'retry_attempt' | 'retry_of' | 'lease_owner' | 'lease_until' | 'callback_secret_enc' | 'platform_token_enc'>>): void {
const entries = Object.entries(redactJobText(patch as Record<string, unknown>)).filter(([, value]) => value !== undefined)
if (entries.length === 0) return
const sets: string[] = []
Expand Down
54 changes: 50 additions & 4 deletions src/jobs/service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
import { GithubApp } from '../repos/app.js'
import { parseIssueUrl, parsePrUrl } from './prUrl.js'
import { buildCarryOver, buildRunDigest, decideNextJob, failureStreak, lastDurationMs, MIN_DISCOVERY_INTERVAL_MINUTES, parseCarryOverLink, type CarryOverReader } from './continuity.js'
import { isTemporalStatus, summariseDelta, type TemporalDelta, type TemporalStatus } from '../lib/temporal.js'
import { planTick, scheduleFromRow, type TickPlan } from '../schedules/tick.js'
import { utcDayEnd, utcDayStart } from '../schedules/budget.js'
import type { ScheduleRow } from '../db.js'
Expand Down Expand Up @@ -896,7 +897,7 @@
}
}

private async preflight(repo: RepoRow, baseBranch: string, trigger: string): Promise<{ ok: boolean; baseSha?: string; warning?: string; error?: string; code?: EnqueueRefusalCode; detail?: string }> {
private async preflight(repo: RepoRow, baseBranch: string, trigger: string): Promise<{ ok: boolean; baseSha?: string; warning?: string; temporal?: TemporalDelta; error?: string; code?: EnqueueRefusalCode; detail?: string }> {
if (this.runner.name !== 'flybot') return { ok: true }
try {
const token = await repoGithubToken(repo, this.config, this.githubApp, this.store)
Expand All @@ -918,7 +919,8 @@
else if (forkBranch.sha !== branch.sha) warning = 'fork is behind upstream'
}
}
return { ok: true, baseSha: branch.sha, warning }
const temporal = await this.temporalDelta(client, repo.id, upstreamOwner, upstreamName, branch.sha)
return { ok: true, baseSha: branch.sha, warning, temporal }
} catch (error) {
if (error instanceof GithubError && (error.status === 401 || error.status === 403) && !/rate limit/i.test(error.message)) {
this.markCredentialInvalid(repo, error)
Expand All @@ -928,6 +930,38 @@
}
}

// #366 slice 1: what changed between the previous finished run's base and this
// one. Two bounded reads, and every way they can end short is `partial` rather
// than a number the goal would present as complete.
private async temporalDelta(client: GithubClient, repoId: string, owner: string, name: string, baseSha: string): Promise<TemporalDelta> {
// The newest finished job that recorded a base, asked as one query rather
// than as a page of the newest rows: on a busy repository the newest twenty
// can all be unfinished, and that must not read as a first run. A rerun at
// the same base is still a previous run and says `no new commits`, which is
// a fact worth having; only a repository with no such job has none.
const sinceSha = this.store.latestFinishedBaseSha(repoId)
if (sinceSha === null) return { sinceSha: null, status: 'ahead', aheadBy: 0, mergedPrs: [], complete: true }
let status: TemporalStatus = 'unknown'
let aheadBy = 0
try {
const compare = await client.compareCommits(owner, name, sinceSha, baseSha)
status = isTemporalStatus(compare.status) ? compare.status : 'unknown'
aheadBy = status === 'unknown' ? 0 : Math.max(0, compare.aheadBy)
} catch (error) {
log.warn('temporal compare failed', { repo: repoId, error: String(error) })
return { sinceSha, status: 'unknown', aheadBy: 0, mergedPrs: [], complete: false }
}
try {
const previous = await client.getCommitMeta(owner, name, sinceSha)
if (previous === null) return { sinceSha, status, aheadBy, mergedPrs: [], complete: false }
const merged = await client.listMergedPullRequestsSince(owner, name, previous.committedAt)
return { sinceSha, status, aheadBy, mergedPrs: merged.pulls.map((pull) => pull.number), complete: merged.complete }
} catch (error) {
log.warn('temporal merged-PR read failed', { repo: repoId, error: String(error) })
return { sinceSha, status, aheadBy, mergedPrs: [], complete: false }
}
}

private async checkCredential(repo: RepoRow): Promise<{ ok: boolean; code?: EnqueueRefusalCode; error?: string }> {
const cached = this.credentialCache.get(repo.id)
if (cached !== undefined && Date.now() - cached.at < 10 * 60_000) return { ok: true }
Expand Down Expand Up @@ -1074,11 +1108,13 @@
}
let preflightSha: string | null = null
let preflightWarning = ''
let temporal: TemporalDelta | undefined
if (kind === 'iterate') {
const preflight = await this.preflight(repo, baseBranch, trigger)
if (!preflight.ok) return { ok: false, error: preflight.error, code: preflight.code, detail: preflight.detail }
preflightSha = preflight.baseSha ?? null
preflightWarning = preflight.warning ?? ''
temporal = preflight.temporal
}
const baseGoal = kind === 'review' ? DEFAULT_REVIEW_GOAL : kind === 'discover' ? DEFAULT_DISCOVER_GOAL : kind === 'triage' ? DEFAULT_TRIAGE_GOAL : kind === 'scope' ? DEFAULT_SCOPE_GOAL : kind === 'verify' ? DEFAULT_VERIFY_GOAL : (repo.goal !== '' ? repo.goal : DEFAULT_GOAL)
const rawGoal = goalOverride ?? baseGoal
Expand All @@ -1092,8 +1128,14 @@
read: (this.carryOverReaderOverride ?? ((target: RepoRow) => this.carryOverReader(target)))(repo),
})).text
: ''
const goal = ((preflightWarning === '' ? rawGoal : `${rawGoal}\n\nPreflight: ${preflightWarning}. Sync the fork before creating the working branch.`)
+ (carryOver === '' ? '' : `\n\n${carryOver}`)).slice(0, 4000)
// #366 slice 1: exactly one data line, and only when a previous run exists.
// The line is placed before the goal is capped: appending it after the cap
// would let a goal that is already near 4000 characters swallow it silently,
// so the body is what gets truncated and the data lines keep their room.
const temporalLine = temporal === undefined ? '' : summariseDelta(temporal)
const dataTail = (temporalLine === '' ? '' : `\n\n${temporalLine}`) + (carryOver === '' ? '' : `\n\n${carryOver}`)
const goalBody = preflightWarning === '' ? rawGoal : `${rawGoal}\n\nPreflight: ${preflightWarning}. Sync the fork before creating the working branch.`
const goal = `${goalBody.slice(0, Math.max(0, 4000 - dataTail.length))}${dataTail}`.slice(0, 4000)
const continuityUsed = carryOver !== ''
const jobId = `job_${randomToken(10)}`
const callbackSecret = randomToken(24)
Expand Down Expand Up @@ -1194,6 +1236,10 @@
throw error
}
if (continuityUsed) this.store.updateJob(job.id, { continuity_used: 1 })
if (temporal !== undefined) {
job.temporal_json = JSON.stringify(temporal)
this.store.updateJob(job.id, { temporal_json: job.temporal_json })
}
if (rules !== undefined) {
this.store.saveJobContext({
jobId: job.id,
Expand Down
68 changes: 68 additions & 0 deletions src/lib/temporal.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
// #366 slice 1: the temporal contract. A job's goal carries one line saying what
// happened between the previous run's base and this run's base. The wording is
// decided here, in a pure function, so the same inputs always produce the same
// line and the reads cannot quietly change it.
//
// The rule the line exists for: a read that failed or hit its page cap is
// reported as `partial`, never as the smaller number it managed to see. A job
// that is told "2 merged PRs" when the walk stopped at two pages would rather
// not be told anything than be told a number it can trust.

export type TemporalStatus = 'ahead' | 'identical' | 'behind' | 'diverged' | 'unknown'

export interface TemporalDelta {
// `base_sha` of the newest finished job for the same repository, `null` for a
// first run. Never the current job's own base.
sinceSha: string | null
// `compareCommits(sinceSha, baseSha).status`, or `unknown` when that read failed.
status: TemporalStatus
aheadBy: number
// Merged pull request numbers inside the window, deduplicated by the caller or
// here; `complete` is false when the page cap was reached or a read failed.
mergedPrs: number[]
complete: boolean
}

const STATUSES: readonly TemporalStatus[] = ['ahead', 'identical', 'behind', 'diverged', 'unknown']

// GitHub returns `status: 'unknown'` for a compare it could not resolve, so the
// name alone does not prove the read worked; anything not in the list is treated
// the same way rather than guessed at.
export function isTemporalStatus(value: string): value is TemporalStatus {
return (STATUSES as readonly string[]).includes(value)
}

const SHORT_SHA = 7
const LISTED_PRS = 5

export function summariseSince(sinceSha: string | null): string {
return sinceSha === null ? 'Since' : `Since ${sinceSha.slice(0, SHORT_SHA)}`
}

function plural(count: number, one: string, many: string): string {
return count === 1 ? one : many
}

function describeCommits(delta: TemporalDelta): string {
if (delta.status === 'identical') return 'no new commits'
if (delta.status === 'behind') return 'base is behind the previous run'
if (delta.status === 'diverged') return 'diverged from the previous run'
return `${String(delta.aheadBy)} ${plural(delta.aheadBy, 'commit', 'commits')} ahead`
}

function describePullRequests(delta: TemporalDelta): string {
if (!delta.complete) return 'merged PRs partial'
const numbers = [...new Set(delta.mergedPrs)].sort((a, b) => a - b)
const count = numbers.length
const head = `${String(count)} ${plural(count, 'merged PR', 'merged PRs')}`
if (count === 0) return head
const shown = numbers.slice(0, LISTED_PRS).map((number) => `#${String(number)}`)
const tail = count > LISTED_PRS ? `${shown.join(', ')}, …` : shown.join(', ')
return `${head} (${tail})`
}

export function summariseDelta(delta: TemporalDelta): string {
if (delta.sinceSha === null) return 'Since: no previous run'
if (delta.status === 'unknown') return `${summariseSince(delta.sinceSha)}: delta unavailable (partial)`
return `${summariseSince(delta.sinceSha)}: ${describeCommits(delta)}, ${describePullRequests(delta)}`
}
Loading
Loading