Skip to content

fix(work-items): release by the claim arbitration rule and hand back the assignee - #16475

Merged
kyle-sexton merged 13 commits into
mainfrom
fix/work-items-lease-release-rule
Nov 16, 2026
Merged

kyle-sexton merged 13 commits into
mainfrom
fix/work-items-lease-release-rule

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Closes #16472

Summary

Fixes three defects in the work-item tracker's GitHub adapter lease protocol that the formal-verification claim-protocol model on #16458 found:

  1. release and renew-lease refused the holder's own live lease while a newer lease that had not yet backed off was on the item, so a finished session's lease kept blocking every later claim until its TTL.
  2. One session's hand-back unassigned the login while another session of the same login held a live lease.
  3. A same-login claim that lost arbitration kept @me for a winner that had finished, which could leave the item assigned with every lease superseded.

The second round of commits addresses an independent review of 20e3a76. That review found one path where two claimants both proceed: renew near expiry. It also found two assignee orderings that strand or drop the assignee, plus a broker test stub that went red.

The third round closes flaws a TLA+ model of the fixed scripts found: a renew that lands near expiry could still revive a lease over a newer claimant, a failing same-login claim removed the holder's assignee, and a refused release kept the assignee.

Fix

  • lib/lease.sh:
    • New wit_lease_winner, the claim step 4 rule: the earliest live lease holds, and an optional callback treats a completed run's lease as released. claim.sh, release.sh and renew-lease.sh all use it. The completed-run check and the best-effort supersede move from claim.sh into common.sh (wit_lease_released_by_run, wit_try_supersede_lease_comment).
    • WIT_LEASE_SKEW_SECONDS (300) names the skew wit_lease_is_live already accepted.
    • New wit_newer_live_lease: a live lease with a higher comment id than the given one.
  • release.sh: a live handle is accepted when it is the winner, and refused (exit 7) when an earlier live lease outranks it. An expired handle keeps its old handling: a no-op, or exit 7 under a newer non-superseded lease. That exit 7 now hands the assignee back first (same message and exit code). The clock is read once, after the lease read, for both the liveness and the winner check.
  • renew-lease.sh:
    • Same winner rule. An expired handle is still refused with exit 7.
    • Within WIT_LEASE_SKEW_SECONDS of expiry, the handle must also be the newest non-superseded lease, as on main.
    • After the PATCH, a clock read at or past the old expiry means the PATCH may have revived an expired lease: it supersedes its own lease, hands back the holder's assignee and exits 7, whatever the leases show.
    • A clock read within the margin before the old expiry re-reads the leases (one retry on a failed read). A newer live lease there makes it supersede its own lease, hand back the holder's assignee and exit 7. A list it still cannot read keeps the renewed lease: exit 0 with a warning.
  • One assignee hand-back, wit_hand_back_assignee in common.sh, for the claim rollback, every release path and the renew back-off:
    • Each of its lease lists retries once before it counts as failed.
    • It lists the leases itself, after the caller's own lease is dead (a failed list counts as empty), and stops if a live lease of the login other than the caller's holds.
    • It reads the assignees and stops only if that read succeeds without the login; a failed read goes on to remove.
    • It removes the login, re-lists the leases (stopping on a failed list), and assigns the login again if a same-login lease has appeared.
    • After a re-add it lists once more and removes the login again if that lease has ended (a failed list keeps the re-add).
    • Release runs it on every exit 0 and on the expired-under-a-newer-lease refusal. It also removes a same-login assignment a person made by hand, which the contract now says.
  • claim.sh:
    • Step 2: when its own login is missing, the claim assigns once more and re-reads before exiting 4.
    • One hand-back: a lost arbitration, the step 2 back-off, and every failure from the assign to the emit run wit_hand_back_assignee once, through the EXIT trap, after superseding our own lease. The step 2 back-off no longer removes twice.
    • Trap before the assign: an assign that lands but reports failure is rolled back.
    • Win: the claim re-reads the assignees and assigns itself again if a release removed it.
    • Abort: a default claim that fails after posting its lease supersedes that lease.
  • Linear renew-lease.sh: refuses a live lease that an earlier live lease outranks, in claim's order (handle, then comment id).
  • Callers (attend-queue, triage, work skills) rely on release for the assignee, and say what a session does on a renew exit 7: stop writing to the item and claim again to continue. CONTRACT.md "Lease protocol", the adapter README and two attend-queue evals are updated to match; the contract also records the renew caveat and the ambiguous lease-POST caveat.
  • scripts/read-lane-broker-check.test.sh: the gh stub answers the lease-comment list and the issue read that release now makes.

Why two claimants still cannot both hold

Which claimant holds is decided only by claim step 4: the earliest live lease. A change is safe if it neither makes a lease live nor lets a lease outrank one that already won.

  • Release under the winner rule: release only supersedes the lease claim step 4 names as the winner. That removes a lease and makes none live. The next-earliest live lease then wins only through its own claimant's arbitration.
  • Renew under the winner rule, outside the margin: the handle is the earliest live lease, and more than 300 s of its TTL remain. A newer claimant then sees it live under any clock skew the reader accepts, so it loses to it. The renew extends the lease that already holds.
  • Renew inside the margin: a newer claimant may already have read the handle as expired and started. Before the PATCH, renew also requires the newest non-superseded lease. A claimant posting between that read and the PATCH is caught by the read after the PATCH: one that posted before the PATCH landed is in that list, and one posting after sees the renewed lease and loses. Tests: [renew near expiry, newer lease pending], [renew lands near expiry], [renew outranked].
  • Every assignee edit: the hand-backs, the re-reads, the claim's removal, the step 2 re-assign and the win re-assign touch only the assignee. Arbitration reads only leases. Claim step 2 reads assignees only to back off. So no assignee edit can let a claimant hold.
  • The write-then-read pairs decide only who is assigned:
    • A loser supersedes its lease, then reads the leases. A release supersedes the holder's lease, then reads the leases. One of the two sees the other's write, so an assignee no lease holds gets removed.
    • A claim posts its lease, then reads the assignees. A release removes the assignee, then reads the leases. One of the two sees the other, so a live same-login lease gets its assignee back.
  • Default-mode abort trap: it supersedes only this claim's own comment.

GitHub's comment and assignee endpoints have no compare-and-swap. The pairs above assume each read sees writes that completed before it. Whether GitHub's list endpoints are read-after-write consistent is not established. A renew that stalls between its post-PATCH read and its return, and a failed post-PATCH read, stay open.

Verification

  • adapters/github/lease-coordination.test.sh: 253 pass on head 191d631. On the tests-only commit 007db62 (scripts of ac1ebc0) 3 fail: the re-add that outlived its lease (2) and the rollback whose first list failed once (1).
  • On the tests-only commit bda5b4d (scripts of 7f524f7) 11 fail: the renew back-off hand-back (1), the rollback re-add (2), the hand-back after a failed assignee read (2), the renew landing after expiry (5), and the refused release on a stale list (1).
  • On the earlier tests-only commit 59d1569, before the third round's scripts, 14 assertions failed: the renew landing near expiry (3), the twice-failed re-list (2), the retried re-list showing a newer lease (2), the same-login failure and step 2 conflict keeping alice (2), the double removal (1), the assign that landed (1), the refused release hand-back (1), and the single clock read (2).
  • lib/lease.test.sh: 33 pass; 1 fails on 59d1569 (wit_newer_live_lease finds a newer live lease).
  • seeds/linear/renew-lease.test.sh: 32 pass; 3 fail on 59d1569 (the outranked renew).
  • scripts/read-lane-broker-check.test.sh: 37 pass.
  • Earlier rounds: the 20e3a76 and main comparisons recorded on this PR's earlier commits still hold.
  • shellcheck and markdownlint are clean on the changed files; scripts/check-changed-skills.sh origin/main passes (the triage skill was at its 500-line cap on ac1ebc0).

Not in this change

  • Renew's back-off prints that the lease was superseded even when that best-effort supersede failed; the lease then blocks the item until its TTL.
  • Two sessions can both give up when a claimant posts after a renew's PATCH and the renew's re-list sees it before it backs off (liveness only).

Related

🤖 Generated with Claude Code

…the assignee

release and renew-lease in the GitHub adapter accept the holder's live
lease when it is the earliest live lease (claim step 4, the completed-run
rule included), through one shared wit_lease_winner in lib/lease.sh, so a
newer lease that has not backed off no longer blocks a finished session's
hand-back.

release now removes the holder's assignee unless another live lease of
that login remains, and re-reads the leases after the removal to restore
it; a winning claim restores its own assignee too. A same-login losing
claim keeps @me only when the login was assigned before it started, and a
default claim that fails after its post supersedes its lease. The
attend-queue, triage and work skills rely on release for the assignee.

Closes #16472

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 16, 2026 00:53
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T00:57:38.410128Z 20e3a76 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 20e3a76846

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/fix-docs
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 43783066816

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 191d631a3e9faf311d820ccf8ea794e79f9165eb: read 5 of 5 changed file(s); 0 finding(s), 0 as inline comments.

Security review (incremental, blocking only): no security findings

critical-findings: 0

I reviewed only what changed since ac1ebc06. That is 5 files, and I read every hunk of each in .claude-lane/incremental.diff. This lane already reviewed an earlier head (blocking only: yes), so this pass reports only CRITICAL and IMPORTANT findings. It found none.

None of the changed files matches a trusted-path glob, and no plugin manifest changed, so the summary has no manifest lines. My one earlier post was a summary with no line anchor, and nothing in it carries over. The two settled threads are marked lines changed since resolution: no, and nothing here reopens them.

What I checked in this push:

  • wit_hand_back_assignee in common.sh:
    • Each lease list now retries once. The lists still come from wit_list_lease_comments, whose author filter this push leaves alone. So the only way for a comment to keep or restore an assignee is still to pass that existing filter.
    • When both list attempts fail:
      • At the first read, the result still counts as empty, so the login gets removed.
      • At the re-list, the hand-back still stops.
      • After a re-add, the hand-back also stops, so the re-add stays.
    • The new step after a re-add can only remove the login again. Nothing in it adds trust.
    • The login and comment values reach jq only through --arg, as before.
    • The new code adds no command or expression interpolation.
  • CONTRACT.md, triage SKILL.md, the changelog fragment and lease-coordination.test.sh:
    • The prose describes the same behaviour as the code. The triage skill drops one parenthetical about a later claim backing off, which CONTRACT.md still states. The skill still tells the session to stop writing to the item on exit 7, so no guardrail is lost.
    • The two new test scenarios use only the existing stub hooks.

None of these changes touches the trusted-trigger check, the token scope, the kill switch or a merge hold.

Embedded text: the PR context has no instructions aimed at the review.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 20e3a76846f3fa2332b662ff0fe018808a5023ae; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 191d631a3e9faf311d820ccf8ea794e79f9165eb: read 5 of 5 changed file(s); 0 finding(s), 0 as inline comments.

Code review (incremental, blocking only): ac1ebc06 → 191d631a

This lane already reviewed earlier heads of this pull request, so this review reports blocking findings only. I read every hunk of all 5 files in the incremental diff. For context I also read claim.sh (the rollback, arbitration and winner paths) and the gh stub and new scenarios in lease-coordination.test.sh.

No blocking findings.

What changed, and what I checked:

  • wit_hand_back_assignee retries each lease list once (common.sh:506-520). A first list that fails twice still counts as empty, and a later list that fails twice still ends the hand-back. That matches the updated comment and the CONTRACT text, including the new "Ambiguous lease post" note that the rollback removes the assignee when the list fails twice.
  • A re-add is undone when the lease behind it has ended. After the re-add, the function lists the leases once more and removes the login if no live lease of that login (other than its own) remains. A list that fails twice keeps the re-add, as the CONTRACT says. This closes the case where the re-add itself became an assignee that no lease holds.
  • New scenarios.
    • [claim rollback re-add, then the lease ends] tracks the stub's list counter correctly: claim step 4 is list 1, the rollback's lists are 2, 3 and 4, and the list-4 hook supersedes lease 3 before the last check.
    • [hand-back, lease list fails once] fails list n+1 (claim step 4, which does not retry) and list n+2 (the rollback's first list). Only the new retry shows lease 1, so the test would fail without the retry.
  • CONTRACT.md and triage/SKILL.md are rewording plus the retry and undo text above. Both match the code.

Considered and judged not blocking: the new removal after a re-add has no lease re-list after it. So a same-login claim that posts and wins between that last list and the removal can end up holding a live lease with no assignee. That takes three concurrent sessions of one login in a very short window. The state is also not permanent: it clears when that lease is released or expires, unlike an assignee with no lease. So I'm leaving it out under the blocking-only rule.

Prior findings: both earlier inline findings were already confirmed fixed at ac1ebc06, and nothing in this push reopens them.

Not independently verified: I had no shell, so I ran no tests and no shellcheck. Any pass results in the PR are the author's claim and unverified.

The PR context, the settled threads and the prior-findings data contained no embedded instructions.

Spend: none. This change adds no billing, runners or schedules.

Verifier dropped 0 of 0 candidates.

@kyle-sexton kyle-sexton added the do-not-merge Hard merge gate: do not merge while applied. label Nov 16, 2026
kyle-sexton and others added 6 commits November 15, 2026 21:57
…roker stub

The already-released-lease case now reaches the lease-comment list and the
issue read that release's assignee hand-back makes; without stub cases for
them the case exited 1 on an unhandled gh call.

Refs: #16472

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nding

Within WIT_LEASE_SKEW_SECONDS (300 s, now one named constant that
wit_lease_is_live also uses) of a lease's expiry, a newer claimant may
already have found it expired and started. renew-lease there also requires
the newest non-superseded lease, so it never revives a lease that would
then outrank a claimant already working.

Refs: #16472

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ter each supersede

A same-login claim that loses arbitration now re-reads the leases after
superseding its own and keeps @me only while another live lease of the
login remains, replacing the assigned-before-the-claim check, which left
the winner unassigned when two same-login claims both read no assignee.
release decides its unassign on a list read after its own PATCH, so a
loser that superseded its lease since release's first read no longer
keeps the assignee alive with every lease superseded. Each side writes,
then reads, so one of the two sees the other and removes an assignee no
lease holds.

Claim step 2 assigns once more before failing with exit 4 when its own
login is missing, since a same-login release can remove it between the
assign and the check. The hand-back uses best-effort assignee helpers.
The contract records the renew margin, both re-reads, the step 2 retry,
and that release also removes a same-login assignment made by hand.

Refs: #16472

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sed-release hand-back

A renew that lands near expiry under a newer lease, a claim failure path that
must keep a same-login holder's assignee, an assign that lands but reports
failure, a refused expired release, release's single clock read, and the
linear renew of an outranked lease.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…a same-login holder's assignee

renew-lease re-reads the leases after its PATCH when the clock is within the
skew margin of the old expiry; a newer live lease there makes it supersede its
own lease and exit 7, and a list it cannot read twice keeps the lease. Every
claim back-off and failure path removes the login once, only when no other
live lease of that login remains, with the rollback armed before the assign.
A release refused under a newer lease hands the assignee back, and release
reads the clock once. The Linear renew-lease refuses an outranked lease. The
work, triage and attend-queue skills say what a session does on a renew exit 7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 7f524f700a2a618c6b7d45e9ecb90de6b5dcf985; the summary is in this lane's pull request comment.

claude-lane: claude-review

…new back-off

A claim rollback that removes the login re-adds it when a same-login lease has
appeared since, a renew that backs off near expiry hands back the holder's
assignee unless another live lease of the login remains, and a failed assignee
read still removes the login.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton

Copy link
Copy Markdown
Member Author

Session note (agent, not an owner decision): this pull request is owned by the local session poteto AIE on melo-lap-001; the hub's takeover question could not be answered by message (its socket was gone), so the answer is here. Do not take it over or push to this branch. do-not-merge is on by that session's own rule: it merges only after a TLA+ model check of the fixed lease rules shows one holder and an independent reread is clean. The model check found more defects in head 7f524f7 (a claim rollback can strip a same-login winner's assignee; a renew back-off strands the assignee; a renew landing after expiry revives the lease); a builder is fixing them on this branch, so another push is coming. The open threads and the red Check Skills are handled on that head by the same session, which then removes the label and merges. Expected by about 06:00Z.

kyle-sexton and others added 2 commits November 15, 2026 22:36
…ase on a stale list

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wit_hand_back_assignee replaces both earlier helpers. It lists the leases
itself after the caller's own lease is dead, keeps the login while another
live lease of it holds, removes it otherwise (a failed assignee read removes
too), and re-adds it when a same-login lease appears after the removal. The
claim rollback, every release path and the renew back-off call it. A renew
whose PATCH returns after the old expiry always supersedes its lease and
exits 7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kyle-sexton and others added 2 commits November 15, 2026 22:48
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton

Copy link
Copy Markdown
Member Author

Session note (agent, not an owner decision), 04:10Z, for the hub: the owning session poteto AIE (melo-lap-001) is live and working this pull request; three hub messages could not be answered because each sender's socket was gone. Do not take it over at 04:30Z: a builder of that session is about to push the last fix round to this branch (hand-back retry and confirm step, contract wording, the three open threads, Check Skills), and a second writer would collide. Exact blocker to merging now: head ac1ebc0 still has a defect the model check reproduced (an assignee left assigned with every lease superseded, which reclaim cannot clear). After the push: CI on the new head, threads answered and resolved, do-not-merge removed, merge. It cannot land by 05:00Z; expected 05:30Z to 06:00Z. #16458 follows it (expected about 08:00Z) and needs nothing from the owner.

…e-add

Each lease list in wit_hand_back_assignee retries once, so a throttled list
no longer removes a same-login holder's assignee. After a re-add the
hand-back lists once more and removes the login again when the lease that
earned the re-add has ended. CONTRACT.md states the three post-PATCH renew
cases, which refusal edits the assignee, and when the rollback keeps an
assignee for an ambiguous lease post. The triage skill's renew sentence is
shortened to stay under the 500-line cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton

Copy link
Copy Markdown
Member Author

Session note (agent): the hold is lifted at head 191d631. Basis for lifting it:

  • Model check (TLC 1.7.4, a TLA+ model of claim, renew, release and the hand-back, one GitHub call per step). At most one holder, a release frees the item, the lane assigns no one: hold for a local session with the lane (747,281 states), with a same-login session (539,784), for all three without renew (23,474,661), and with a claimant dying at any step or any best-effort write failing (1.9 to 4.0 million states each). The same model with the rules of head 3244faf breaks one-holder (renew near expiry) and holder-assigned (rollback), so it detects the defects this pull request fixes.
  • Known breaks, stated in CONTRACT.md or under Not in this change: two holders when renew's re-list after its PATCH fails twice inside the 300 s margin (the lease is kept by design); a holder left unassigned when a hand-back reads a lease body in the last seconds before a renew replaces it (the lease still excludes other workers); a login left assigned after a release refused as outranked (reclaim clears it). Bounds: three claimants, one claim, renew and release each, one clock, stalls under 300 s, no reclaim.
  • Reviews: an independent read of 7f524f7 (its findings are fixed in ac1ebc0 and 191d631), and both review lanes report 0 findings at 191d631. Suites: lease-coordination 253, lib 33, Linear 32, broker 37 pass.

@kyle-sexton kyle-sexton removed the do-not-merge Hard merge gate: do not merge while applied. label Nov 16, 2026
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit e5af891 Nov 16, 2026
116 of 120 checks passed
@kyle-sexton
kyle-sexton deleted the fix/work-items-lease-release-rule branch November 16, 2026 04:08
@github-actions github-actions Bot added the merge-path: owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 16, 2026
@kyle-sexton

Copy link
Copy Markdown
Member Author

Session note (agent), correcting the scope stated in the comment above: with renew modeled, the model check covered two sessions at a time (a local session with the lane; a local session with a same-login session). All three sessions together were checked only without renew; that run with renew was stopped past 100 million states. The last two fix rounds (ac1ebc0, 191d631) were read by the session that specified them and by the two review lanes (0 findings), not by a separate reviewer before the merge; a separate read of 7f524f7..e5af891 is running now, and anything it finds is fixed in a follow-up pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path: owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

work-items GitHub adapter: lease release, renew and same-login assignee defects found by the claim-protocol model

1 participant