Skip to content

Older live-state event can silently overwrite a newer one in the same batch #7136

Description

@vipulpandey21

What's happening

PatchKubernetesApplicationLiveState (pkg/app/server/applicationlivestatestore/store.go) applies a batch of Kubernetes resource-state events to a snapshot. Each event carries a version, and the code is supposed to skip any event older than what's already stored:

if ev.SnapshotVersion.IsBefore(*snapshot.Version) {
    continue
}

The problem: snapshot.Version is fetched once at the top of the function and never updated while the loop applies events. So every event in the batch gets compared against that same original version, not against the version the previous event in the loop just moved the state to.

Why it matters

If a batch has two events for the same resource where an older one arrives after a newer one (a redelivered retry is a normal way this can happen), the check above doesn't catch it. The older event silently overwrites the newer state — no error, no log.

How I found it

Traced how piped assigns event versions (pkg/app/piped/livestatestore/kubernetes/appnodes.go) through to how the server applies them. Wrote a test with two out-of-order events for the same resource — the final state ends up on the stale value.

I have a fix and a test ready, opening a PR for it.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions