Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 4 additions & 7 deletions docs/TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,13 @@ should land before the next tag is cut (the last release is v0.55.0). Checked ag

## Before the next release

- [ ] **A withheld apply rides the next edit.** S1 withholds an apply that lands over an edit decided
after the flush mark, and a `Repair` puts the edit back. But the apply did land on the cluster, so
the next edit of that object carries the apply's value for every field it did not change, and the
window commits the whole object under that author's name: the overwritten edit is lost, and the
`Repair` then applies the loss. The failing test is
`TestApplyWindow_AWithheldApplyIsNotCommittedUnderTheNextEdit`
(`internal/git/apply_window_carry_test.go`, skipped; `GITOPS_REVERSER_KNOWN_BUGS=1` runs it).
Nothing open.

## Applier claims and bi-directional editing

- [ ] **Persist the withheld-field record.** The fields a withheld apply reverted are kept in memory
(`internal/git/withheld_fields.go`), so after a restart the next edit of such an object carries the
apply's values into Git again, until the `Repair` lands.
- [ ] **Reopen the render-fidelity gate after a Git repair, automatically.** Today a target on
`RenderMatchesLive=False` clears only on an explicit recheck (the
`reconcile.configbutler.ai/requestedAt` annotation, which restarts every scope; the render
Expand Down
21 changes: 20 additions & 1 deletion docs/design/applier-echoes.md
Original file line number Diff line number Diff line change
Expand Up @@ -606,7 +606,8 @@ mirrors and the applier applies.

## Decided while building

The build departs from the plan above in twelve places, decided on 2026-10-20:
The build departs from the plan above in thirteen places, the first twelve decided on 2026-10-20 and
the last on 2026-10-28:

1. **Siblings do not count for F3.** Whether Git is ahead, and the apply epoch, are per worker: an
apply due on another worker writing the same branch does not hold this worker's resyncs. This
Expand Down Expand Up @@ -672,6 +673,24 @@ The build departs from the plan above in twelve places, decided on 2026-10-20:
attempt is sent for the tip that holds the siblings' writes. A sibling's push is foreign to the
others, as it was when they found it by a fetch: its appliers may apply another cluster. A
sibling's edit made while the apply runs stays unprotected, as it was before this change.
13. **A withheld apply's values are kept out of the next edit.** The plan withheld the apply and left
the rest to the `Repair`, but the apply did land: until the `Repair`, the cluster runs its values,
and the next event of the object, by anyone, carried each one its author did not change. A window
commits whole objects, so that author's commit wrote the apply's values, the edit was lost, and
the `Repair` applied the loss. The worker now keeps, per object, the last version an apply window
or a flush recorded as an edit (the last accepted version), and on a withhold records each field
where the withheld object differs from it: the accepted value and the applier's, either of which
may be absent. Every later event of the object, mirrored or not and the `Repair`'s own landing
included, and every resync snapshot of it, is restored against that record: a field still
holding the applier's value was carried over and is set back to the accepted value; a field
holding any other value was changed by someone and leaves the record; a delete clears it. This is
the desired view of
[`window-field-ownership-and-apply-hold.md`](window-field-ownership-and-apply-hold.md#convergence-is-a-separate-operation),
limited to the fields a withheld apply reverted: commits stay whole objects, and no window owns
fields. Three limits are accepted. A list is one field, so an edit to another item of a list the
apply reverted still carries the reverted item. An author setting a field to the applier's value
cannot be told from carrying it, and is set back. The record lives in memory, so a restart loses
it ([`withheld_fields.go`](../../internal/git/withheld_fields.go)).

## References

Expand Down
5 changes: 4 additions & 1 deletion internal/git/applier_claim.go
Original file line number Diff line number Diff line change
Expand Up @@ -402,6 +402,7 @@ func (l *branchWorkerEventLoop) serviceAppliers() {
defer l.publishClaimStatus()
defer l.publishHeld()
now := time.Now()
defer l.forgetAccepted(now)
l.settleApplyResults(now)
l.followRecordedOperations(now)
converging := l.scheduleConverges(now)
Expand Down Expand Up @@ -660,7 +661,8 @@ func (l *branchWorkerEventLoop) endFlush() {

// recordUnpublishedEdits records in the window of an attempt being sent the edits its commit does
// not hold, which the apply may overwrite: those decided after the flush mark, and the live events
// held for a GitTarget's render fidelity.
// held for a GitTarget's render fidelity. Every live event of a held GitTarget is held, in order, so
// the last one held of an object is its last accepted version.
func (l *branchWorkerEventLoop) recordUnpublishedEdits(state *applierState) {
for _, target := range state.targets {
for key, at := range l.flushEdits[target.ref] {
Expand All @@ -669,6 +671,7 @@ func (l *branchWorkerEventLoop) recordUnpublishedEdits(state *applierState) {
if held := l.held[target.ref]; held != nil {
for _, h := range held.events {
state.window.recordEdit(objectKey(h.event.Identifier), h.event.LastFieldManagedAt)
l.keepAccepted(h.event)
}
}
}
Expand Down
14 changes: 11 additions & 3 deletions internal/git/apply_window.go
Original file line number Diff line number Diff line change
Expand Up @@ -276,13 +276,15 @@ func (state *applierState) covers(event Event) bool {
// and must not be mirrored: the apply itself landing on an object nobody edited (F1), or the apply
// arriving late over a newer API edit (withheld, with a Repair pending). Every other event is
// recorded as an API-side edit in the windows of the appliers that cover it, and in the edits the
// attempts of a flush in progress will hold once they are sent (recordFlushEdit). It is called for
// each live event before it reaches the commit window.
// attempts of a flush in progress will hold once they are sent (recordFlushEdit). An event recorded
// either way is kept as its object's last accepted version, what a withheld apply is compared with
// (withheld_fields.go). It is called for each live event before it reaches the commit window.
func (l *branchWorkerEventLoop) withholdLateApply(event Event) bool {
if len(l.appliers) == 0 {
return false
}
now := time.Now()
recorded := false
for _, ref := range l.sortedAppliers() {
applier := l.appliers[ref]
if !applier.covers(event) {
Expand All @@ -293,6 +295,7 @@ func (l *branchWorkerEventLoop) withholdLateApply(event Event) bool {
if l.judgeInWindow(applier, event, now) {
return true
}
recorded = true
case event.Operation != string(itypes.OperationDelete):
if _, alone := ref.writtenBy(event.LastFieldManagers); alone {
// No apply this worker asked for is landing: an echo after the grace, or an apply
Expand All @@ -304,6 +307,9 @@ func (l *branchWorkerEventLoop) withholdLateApply(event Event) bool {
// Decided after the flush mark, so not in the commit the flush's attempts apply: their windows
// hold it from the start (S1).
l.recordFlushEdit(event)
if recorded || l.flushing {
l.keepAccepted(event)
}
return false
}

Expand Down Expand Up @@ -351,8 +357,10 @@ func (l *branchWorkerEventLoop) judgeInWindow(applier *applierState, event Event
// apply over the edit: events arrive in resourceVersion order, so the edit, recorded
// already, was written first, and mirroring this would revert it in Git and on the
// cluster. Only an undo in that same second is lost (C4 in docs/design/applier-echoes.md).
// An edit still held for render fidelity is repaired once it reaches Git.
// An edit still held for render fidelity is repaired once it reaches Git. The fields it
// reverted are recorded, so the next event of the object does not carry them into Git.
l.recordApplyWindowCollision(applier, window, event, collisionWithheld)
l.recordWithheld(event)
if !l.repairOnRelease(applier.ref, event) {
applier.repair = true
}
Expand Down
Loading
Loading