Skip to content

fix(storage): validate checkpoint source offsets#3

Merged
siriusctrl merged 2 commits into
agent/checkpoint-decoderfrom
review/checkpoint-decoder
Jul 22, 2026
Merged

fix(storage): validate checkpoint source offsets#3
siriusctrl merged 2 commits into
agent/checkpoint-decoderfrom
review/checkpoint-decoder

Conversation

@siriusctrl

Copy link
Copy Markdown
Owner

Review finding

The decoder accepted line_end values that produced gaps or overlaps relative to the prior decoded line. The async forward reader naturally supplies contiguous offsets, but the public crate-internal tail-candidate seam could otherwise publish incorrect raw source offsets and committed boundaries.

Fix

  • track the next expected physical line offset and reject gaps, overlaps, and underflow
  • verify a multi-record checkpoint remains hidden across repeated EOF observations and publishes exactly once after later appends complete it
  • exercise deterministic 1-23 byte read chunks, exact raw order, arbitrary m900 candidate offsets, and a very large declared checkpoint count without preallocation

Independent validation

  • cargo fmt --check
  • cargo check --all-targets
  • cargo clippy --all-targets -- -D warnings
  • cargo test --all-targets: 276 passed, 0 failed, 0 ignored

This PR is based on agent/checkpoint-decoder and should be merged there before PR #2. It does not add epoch, truncate, replacement, or reopen behavior to the forward decoder.

Require every decoded physical line to begin exactly where the previous line ended so tail candidates cannot publish raw records with gaps or overlaps.

Cover deterministic chunk boundaries, large declared counts, and multi-record checkpoints that complete across repeated EOF observations.
Validate the sequence after a complete checkpoint before retaining its first record. This prevents an unrepresentable next sequence from surfacing only after the checkpoint has reached its commit boundary.

Keep commit_pending transactional as a defensive invariant and cover the boundary with a retry test that proves pending and byte offsets remain untouched.
@siriusctrl
siriusctrl marked this pull request as ready for review July 22, 2026 10:15
@siriusctrl
siriusctrl merged commit f192bd5 into agent/checkpoint-decoder Jul 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant