Skip to content

feat(control-plane): integration interface for task sources - #14

Draft
itsmihir wants to merge 21 commits into
mainfrom
feat/integration-interface
Draft

itsmihir wants to merge 21 commits into
mainfrom
feat/integration-interface

Conversation

@itsmihir

Copy link
Copy Markdown
Contributor

What this changes

Moves Linear, GitHub and the HTTP API behind one Integration interface in the control plane, so a new task source plugs in with a registry entry instead of edits across core services.

  • Outbound. Each integration implements deliver(ctx, target, message). Core labels harness events as a reply, progress or failure and routes them by the round's source; the integration decides where, or whether, each one shows up.
  • Intake. Tasks and follow-ups are created only through TaskIntake (assign, restart, followUp). A re-delegation is compared with the stored one under the task's row lock, so concurrent webhooks restart a task at most once.
  • Lifecycle hooks. onTaskStarted, onPrOpened, onRoundFinished and assignee run with timeouts; detached work is drained on shutdown.
  • Registry. app.ts builds { linear, api, github }. Each integration has enabled, an optional webhook (mounted at its existing path) and status() (served at /api/integrations/:name).
  • Reliability. A send that fails before anything went out (rate limit, connection refused, DNS, TLS, token fetch) holds the event and retries it instead of dropping it. Every send and hook is time-bounded, leases are renewed throughout long reconciles, and shutdown is bounded.

Behaviour changes

  • A crashed GitHub comment or review round now posts ⚠️ Something went wrong while working on this. Mention @<bot> in a new comment to retry. instead of nothing. Crashed CI-fix rounds stay silent.
  • Failure notices are posted only for rounds that were actually marked failed, after marking, so a late completion or a restart no longer gets a contradictory "task failed".
  • A Linear token error while dead-lettering a task with a missing workspace no longer leaves its other rounds open and its lease held.
  • The onboarding comment, PR assignee mirroring and the Linear run-archive upload happen once, also when events are replayed.
  • GitHub and Linear sends that fail before going out are held and retried (honouring Retry-After, or the platform's rate-limit reset) instead of dropped. A task's later events wait behind a held one.
  • On a Linear task, GitHub rounds no longer wait on a Linear outage.
  • Task-key lookup is case-insensitive.
  • A duplicate Linear delegation no longer refreshes title, URL and flags; a real re-delegation still does.
  • Linear's "Sandbox provisioned" thought is removed.
  • A disabled integration's messages and hooks are skipped instead of posted or held.
  • Slack notifications are opted in per source with SLACK_NOTIFY_SOURCES (default linear, so nothing changes by default).
  • /api/integrations/linear and /api/integrations/github keep their fields and gain holds: { heldTasks, oldestHoldAgeMs }; /api/integrations/api is new.
  • If Linear's token can't be fetched as a PR opens, that PR isn't auto-assigned.

Upgrade notes

  • Two migrations: a unique index on inbox (source, kind, external_id), then a case-insensitive unique index on tasks (lower(key)) that replaces tasks_key_uniq. They take brief locks; on a lock timeout the new pod restarts and retries while old pods keep serving. See docs/kubernetes/README.md (Upgrade, Database migrations).
  • The control-plane shutdown grace period is a chart value, controlPlane.terminationGracePeriodSeconds (default 150), and Compose's stop_grace_period is 150s.
  • Pods only claim tasks whose source they know, so rolling deploys and rollbacks stay safe.

How I tested it

  • control-plane/: bun run typecheck, bun run biome:ci, and bun test --isolate against a throwaway Postgres 18: 1832 pass.
  • cli/: bun test: 293 pass; tsc --noEmit clean.
  • Chart: bun run test:chart: 164 pass.
  • harness/: make lint clean; make test passes except test_distillation … component_bootstrap, which also fails on main on macOS (/private/var vs /var temp path).
  • The first commits pin the existing behaviour with characterization tests before changing it; later commits add tests for holds, timeouts, hooks, concurrent re-delegation, migrations and the registry wiring.

🤖 Generated with Claude Code

itsmihir and others added 21 commits October 30, 2026 22:14
Pin today's behaviour before moving integrations behind a common
interface: failure notices per round kind for task failures and missing
workspaces, replayed harness events, webhook redeliveries, restarts and
Slack gating. Tests for behaviour the refactor changes on purpose are
marked, so they are flipped knowingly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Stub the rows failTaskInbox marks, so failure tests don't depend on
posting before marking. Cover agent_error on Linear comment rounds and
Slack during a Linear outage, use the GitHub client's real 429 error,
and wait on observable calls instead of fixed sleeps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…deliver

Linear, GitHub and the API each get an integration module with one
deliver(ctx, target, message). Core only labels harness events as a
reply, progress or failure and routes them by the round's source; the
module decides where, or whether, each one shows up.

Failures now mark rows first and notify only the rows actually marked,
so a late completion or a restart can't get a contradictory "task
failed". A crashed GitHub comment or review round now gets a short retry
hint on the PR instead of nothing, and a Linear token error no longer
leaves the task's other rows open and its lease held.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pin that a Linear token failure on pr_opened and on an assignment's
completion waits instead of advancing, and that no retry hint is posted
when GitHub webhooks are off. Rows whose source has no integration are
skipped instead of throwing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A send that fails before anything went out (rate limit, connection
refused, DNS, TLS, token fetch) now holds the event and retries it,
honouring Retry-After, instead of dropping the reply. Every send runs
under a 20 s timeout; a timed-out or failed send is dropped and the task
yields the tick, so one slow platform can't stall the others.

Leases are renewed before each task's reconcile, and pods only claim
tasks whose source they know, which keeps rolling deploys and rollbacks
safe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Linear PR attach, run-archive upload and assignee lookup, and
GitHub's review re-request, move into integration hooks run with
timeouts; detached work is drained on shutdown. Replayed events no
longer repeat the onboarding comment, assignee mirroring or archive
upload, which now run only on the first transition.

The "Sandbox provisioned" thought is gone, and the chart and Compose get
a shutdown grace period covering one task's reconcile plus the drain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The follow-up prompt no longer assumes Linear. A test pins the two
agent_error formats the control plane routes on: a failed assignment's
user-safe summary, and a crashed round's exception text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each cursor advance renews the task's lease, so a run of slow sends
can't outlive it. A rate limit without Retry-After waits for the
platform's reset time, at least 60 s, instead of retrying every tick,
and a GitHub 403 carrying rate-limit headers holds like a 429.

Shutdown stops reconciling and claiming new tasks, the onboarding
comment gets the send timeout, assignee logs carry the account id, and
timeouts, the drain and the hooks get tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bounding the onboarding post made one task's reconcile up to 110s, which
with the 10s drain left the 120s default no headroom.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…index

Task keys become unique regardless of case, and inbox rows dedupe on
(source, kind, external_id), so ids of different kinds from one source
can't collide. The old (source, external_id) index stays for a release
so older pods keep working during a rolling deploy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Linear, the API and GitHub create tasks and follow-ups only through
TaskIntake. A re-delegation is compared with the stored one under the
task's row lock, so concurrent webhooks restart a task at most once, and
a restart refreshes title, URL, flags and requester. A duplicate Linear
delegation now changes nothing.

Follow-ups lock the task, drop rounds for PRs the task no longer owns,
and run the CI loop breaker in the same transaction. Hard-coded source
lists are derived from the task-source enum, with compile-time checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The choices come from the shared type, so a new source can't be
forgotten.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lack opt-in

Integrations carry enabled, webhook and status. Core mounts each enabled
integration's webhook at its existing path, serves
/api/integrations/:name with the existing fields plus held sends, and
skips a disabled integration's sends and hooks instead of holding them.

Slack notifications are opted in per source with SLACK_NOTIFY_SOURCES
(default linear), and mentions come from the integration's assignee
lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Helm values, schema and template, the install target file and the
configuration docs gain it. Unset keeps Linear-only notifications.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Create the inbox index first and drop the old task-key index last, so
the boot migration never holds tasks exclusively while waiting on inbox.
The upgrade docs cover the lock-timeout retry, the duplicate-key failure
and how to find such rows, and what older pods may log during the
overlap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add tests for concurrent re-delegation, disabled integrations on the
failure paths, the app's enabled mapping and status body, and the new
ON CONFLICT target. Restore log fields and messages that dashboards may
key on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Renew every lease the pod holds before each reconcile, including tasks
still provisioning, and never dispatch a task already in flight, so a
long tick can't provision a task twice. Shutdown stops between events,
failure notices go to exactly the rows failTaskInbox marked, and the
drain also waits for work detached while draining.

The upgrade docs no longer suggest dropping the new inbox index after a
rollback: the migration is already recorded, so the next upgrade would
not rebuild it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Linear's issue-state change and Provisioning action move into its
onTaskStarted hook, GitHub's deliver narrows to its own target kinds,
one helper delivers for the dispatcher and both failure paths, failure
notices dedupe per source and target, Slack calls get the send timeout,
and tests share one fake-integration fixture, so a new integration needs
no core edits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TaskIntake rejects a delegation order that isn't a non-negative safe
integer, since NaN would make every redelivery restart the task, and
restart refuses non-API tasks. Also fixes stale test names and comments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Within a tick, renew every lease the pod holds once a quarter of the
lease has passed, checked before each reconcile and each event, and stop
a task's events if its own lease was lost. One long reconcile no longer
lets a sibling pod take over the pod's other tasks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comments and test names say what the code does and why, without citing
documents outside the repo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant