Skip to content

feat(stovepipe): Track issues in bounded passes - #940

Draft
prathshenoy wants to merge 1 commit into
codex/stovepipe-issuetrack-reconcilefrom
codex/stovepipe-issuetrack-delivery
Draft

prathshenoy wants to merge 1 commit into
codex/stovepipe-issuetrack-reconcilefrom
codex/stovepipe-issuetrack-delivery

Conversation

@prathshenoy

@prathshenoy prathshenoy commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Summary

What:

  • Add an issue-tracking controller that opens reported issues and closes missing issues when a build is finalized.
  • Process issues in small batches, resume saved work, and ignore outdated notifications without enabling the consumer yet.

Why:

  • Turn persisted build issues into current issue state without trying to process an entire queue in one delivery. Large issue sets may take longer than a delivery allows, so each pass handles a bounded batch and saves its position before requesting another delivery. The next pass resumes the planned change and scan instead of starting over.
  • Keep delayed messages from undoing newer issue state. A newer notification first finishes any saved work, then checks whether its build is still eligible to update the queue. Refreshes open reported issues without closing missing ones; finalization also closes issues absent from the build’s accumulated failures, except when the build was canceled and validation may not have run.

Test Plan

  • Add unit tests.
  • Verify chunk continuation and stale-worker rejection with two real consumers, forced partition takeover, and MySQL-backed state and history.

Revert Plan

  • Revert dependents first, then this PR. No runtime consumer is enabled; the planned-change application and storage layers remain in place.

Issue Links

Stack

  1. feat(stovepipe): Retain versioned issue state #942
  2. feat(stovepipe): Save planned issue changes #943
  3. feat(stovepipe): Apply planned issue changes #932
  4. feat(stovepipe): Track issues in bounded passes #940
  5. feat(stovepipe): Publish issue-tracking updates #939

@prathshenoy
prathshenoy force-pushed the codex/stovepipe-issuetrack-delivery branch from 9a6c001 to d37bfce Compare October 26, 2026 02:26
@prathshenoy
prathshenoy added this pull request to stack #933 October 26, 2026 16:04
@prathshenoy
prathshenoy force-pushed the codex/stovepipe-issuetrack-delivery branch 2 times, most recently from e8a500b to 88d2243 Compare October 27, 2026 01:20
@prathshenoy
prathshenoy force-pushed the codex/stovepipe-issuetrack-reconcile branch from 7e6af1e to 9721cf1 Compare October 27, 2026 01:20
@prathshenoy prathshenoy changed the title feat(stovepipe): Orchestrate issue checkpoints feat(stovepipe): Track issues in bounded passes Oct 27, 2026
@prathshenoy
prathshenoy removed this pull request from stack #933 October 27, 2026 01:26
@prathshenoy
prathshenoy added this pull request to stack #944 October 27, 2026 01:26
@prathshenoy
prathshenoy force-pushed the codex/stovepipe-issuetrack-reconcile branch from 9721cf1 to a14c3cd Compare October 27, 2026 03:02
@prathshenoy
prathshenoy force-pushed the codex/stovepipe-issuetrack-delivery branch 2 times, most recently from 39b4dc0 to a0aeceb Compare October 27, 2026 03:10
@prathshenoy
prathshenoy force-pushed the codex/stovepipe-issuetrack-reconcile branch 2 times, most recently from 93104e6 to 802fea3 Compare October 27, 2026 03:24
@prathshenoy
prathshenoy force-pushed the codex/stovepipe-issuetrack-delivery branch from a0aeceb to 5422e9a Compare October 27, 2026 03:24
@prathshenoy
prathshenoy force-pushed the codex/stovepipe-issuetrack-delivery branch from 5422e9a to 03e6db1 Compare October 27, 2026 03:38
@prathshenoy
prathshenoy force-pushed the codex/stovepipe-issuetrack-reconcile branch from 802fea3 to 09f2749 Compare October 27, 2026 03:38
DivyanshGarg380 pushed a commit to DivyanshGarg380/submitqueue that referenced this pull request Oct 27, 2026
## Summary

**What**:

- Persist queue-scoped source ordering, reconciliation ownership, and
exclusive issue cursors with guarded progress updates.
- Keep storage point-keyed and leave runtime writers disabled.

**Why**:

- Resume interrupted batches without restarting their scans or losing
the boundary after the last open issue disappears.
- Distinguish the notification that owns a pass from another
notification helping finish it.

## Test Plan

- [x] Add unit tests.
- [x] Verify checkpoint persistence, queue isolation, identifier
boundaries, and competing guarded updates against MySQL.

## Revert Plan

- Revert dependents first, then this PR. No runtime writer is enabled;
the additive progress table can remain unused.

## Issue Links

- [CODEM-607](https://linear.app/uber/issue/CODEM-607/build-analysis)

## Stack

1. uber#930
1. uber#932
1. uber#940
1. uber#939
@prathshenoy
prathshenoy removed this pull request from stack #944 October 27, 2026 15:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant