Repository navigation
feat(stovepipe): Track issues in bounded passes - #950
Merged
Merged
Conversation
This was referenced Oct 27, 2026
2 tasks done
prathshenoy
force-pushed
the
codex/stovepipe-row-controller
branch
from
October 27, 2026 16:42
1706ec9 to
5a991d3
Compare
prathshenoy
marked this pull request as ready for review
October 27, 2026 16:56
prathshenoy
requested review from
a team,
behinddwalls and
sbalabanov
as code owners
October 27, 2026 16:56
prathshenoy
force-pushed
the
codex/stovepipe-row-controller
branch
from
October 27, 2026 17:12
5a991d3 to
acb404d
Compare
mnoah1
reviewed
Oct 27, 2026
2 tasks done
mnoah1
approved these changes
Oct 27, 2026
2 tasks done
2 tasks done
DivyanshGarg380
pushed a commit
to DivyanshGarg380/submitqueue
that referenced
this pull request
Oct 28, 2026
## Summary **What**: - Retain each issue’s open or closed status and version instead of deleting it when it closes. - Keep repair selection limited to open issues and allow issue records to be read in batches. **Why**: - Keep an issue’s version after it closes so an old worker cannot treat the missing row as permission to create it again. Deleting closed issues erases that protection; retaining the row lets later writes be checked against the version they originally read. - Provide a direct way to read whether an issue is open without reconstructing its state from past builds or history. Keep those reads and updates compatible with plain per-key storage, without requiring transactions across records or a separate index of open issues. ## Test Plan - [x] Add unit tests. - [x] Verify closure, reopening, competing version guards, queue isolation, and mixed-status scans against MySQL. ## Revert Plan - Revert dependents first, then this PR. No issue-tracking consumer is enabled; the additive state table can remain unused. Deployments that enable writers must coordinate any rollback or backfill before restoring earlier readers. ## Issue Links - [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis) ## Stack 1. uber#942 1. uber#948 1. uber#949 1. uber#950 1. uber#951
DivyanshGarg380
pushed a commit
to DivyanshGarg380/submitqueue
that referenced
this pull request
Oct 28, 2026
## Summary **What**: - Record the last build processed for each issue and preserve the build and time of its latest opening or closing. - Store a resumable current-issue cursor for each queue. **Why**: - Give later processing enough information to recognize work that already succeeded. An issue update and its history entry are separate writes; retaining the actual transition lets a retry finish the history without inventing a new event or timestamp. - Prepare for rejecting delayed writes without storing instructions for a future change. Even unchanged results need a processed-build marker so the next layer can distinguish an old result from the current one. ## Test Plan - [x] Add unit tests. - [x] Verify issue state and cursor persistence against a real MySQL database. ## Revert Plan - Revert dependent PRs first, then this PR. Issue-tracking consumption remains disabled, so reverting does not interrupt an active issue-tracking worker. ## Issue Links - [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis) ## Stack 1. uber#942 1. uber#948 1. uber#949 1. uber#950 1. uber#951
DivyanshGarg380
pushed a commit
to DivyanshGarg380/submitqueue
that referenced
this pull request
Oct 28, 2026
## Summary **What**: - Apply one issue’s open or closed status and record its transition history with retry-safe conditional writes. **Why**: - Keep partial failures recoverable without a transaction across records. If the issue changes but recording history fails, a retry can use the retained cursor and transition facts to finish the same history entry. - Reject work from a worker that resumes after another worker has moved on. Check the queue revision after reading the issue and retain closed rows so an old write cannot recreate a missing opening. ## Test Plan - [x] Add unit tests. ## Revert Plan - Revert dependent PRs first, then this PR. Issue-tracking consumption remains disabled, so reverting does not interrupt an active issue-tracking worker. ## Issue Links - [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis) ## Stack 1. uber#942 1. uber#948 1. uber#949 1. uber#950 1. uber#951
DivyanshGarg380
pushed a commit
to DivyanshGarg380/submitqueue
that referenced
this pull request
Oct 28, 2026
## Summary **What**: - Publish issue-tracking refresh and finalization notifications after build analysis persists its results. - Add the publication route without enabling issue-tracking consumption. **Why**: - Give issue tracking a reliable handoff from build analysis. Publishing after persistence ensures it can read the build’s recorded issues, and a failed publish can retry without losing the handoff. Require a complete cumulative terminal result so finalization does not depend on an earlier worker finishing its writes. - Keep terminal recovery separate from this rollout. Registering only the publication route allows the processing layers to be reviewed without starting a consumer whose recovery policy is not yet wired. ## Test Plan - [x] Add unit tests. ## Revert Plan - Revert dependent PRs first, then this PR. Issue-tracking consumption remains disabled, so reverting does not interrupt an active issue-tracking worker. ## Issue Links - [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis) ## Stack 1. uber#942 1. uber#948 1. uber#949 1. uber#950 1. uber#951 1. uber#958 1. uber#961
DivyanshGarg380
pushed a commit
to DivyanshGarg380/submitqueue
that referenced
this pull request
Oct 28, 2026
## Summary **What**: - Keep existing issues open when any task-wide failure is recorded for a build. - Apply all reported issues before reconciling absent ones, using the existing processing phases. **Why**: - Avoid treating missing target results as recovery. A task-wide failure can mean validation never ran or could not identify failed units, so closing earlier issues would claim recovery without evidence. - Avoid a separate precheck and new storage operations for data we already need to read. Recording task-wide failures while applying reported issues lets the closing pass use a decision established over the whole build. ## Test Plan - [x] Add unit tests. - [x] Verify terminal intent and task-wide failure information persist against a real MySQL database. ## Revert Plan - Revert this PR before enabling issue-tracking consumption. The added columns can remain unused; once consumption is enabled, drain active passes before rolling back. ## Issue Links - [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis) ## Stack 1. uber#942 1. uber#948 1. uber#949 1. uber#950 1. uber#951 1. uber#958 1. uber#961
DivyanshGarg380
pushed a commit
to DivyanshGarg380/submitqueue
that referenced
this pull request
Oct 28, 2026
## Summary **What**: - Add dead-letter recovery that stops failed issue-tracking passes while keeping completed issue changes. - Record incomplete issue tracking without changing the build's validation outcome. **Why**: - Unblock later builds when issue tracking exhausts normal retries. Today the unfinished pass remains saved, so later notifications try to resume the same failed work. Recovery needs to stop that pass without throwing away issue changes that already succeeded. - Keep delayed workers from changing issue state after recovery. Simply clearing progress would let a paused worker write an old opening or closing after a newer build has run. Recovery protects the current issue and retains its history before allowing newer work through. ## Test Plan - [x] Add unit tests. - [x] Verify recovery progress persists against a real MySQL database. ## Revert Plan - Revert this PR before enabling consumption. If already enabled, stop consumers and reconcile recovery progress before rolling back so an older consumer does not resume abandoned work. ## Issue Links - [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis) ## Stack 1. uber#942 1. uber#948 1. uber#949 1. uber#950 1. uber#951 1. uber#958 1. uber#961 1. uber#962
DivyanshGarg380
pushed a commit
to DivyanshGarg380/submitqueue
that referenced
this pull request
Oct 28, 2026
## Summary **What**: - Enable normal issue tracking and dead-letter recovery in the reference service. **Why**: - Consume the issue notifications that build analysis already publishes. Until now they accumulated on the queue without updating current issues or their history. Registering both consumers makes that path operational, while deployments can still supply their own build-issue detector. - Keep failed tracking from blocking later builds indefinitely. Normal processing uses bounded retries, and recovery stops incomplete work safely instead of repeating the same failed pass forever. ## Test Plan - [x] Add unit tests. - [x] Verify analysis handoff and dead-letter recovery through the running service and MySQL. ## Revert Plan - Revert this PR to disable both consumers without deleting persisted issue state, history, or progress. ## Issue Links - [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis) ## Stack 1. uber#942 1. uber#948 1. uber#949 1. uber#950 1. uber#951 1. uber#958 1. uber#961 1. uber#962
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
What:
Why:
Test Plan
Revert Plan
Issue Links
Stack