Skip to content

Trimmed the Redis streams by retention on every append. - #46

Open
moedash wants to merge 4 commits into
moe/AI-198-if-pyext-9-redis-providerfrom
moe/AI-198-if-pyext-10-redis-trimming
Open

moedash wants to merge 4 commits into
moe/AI-198-if-pyext-9-redis-providerfrom
moe/AI-198-if-pyext-10-redis-trimming

Conversation

@moedash

@moedash moedash commented Oct 3, 2026

Copy link
Copy Markdown
Owner

This PR trims the Redis provider's logs by age, and optionally by count, on every append.

What changed?

  • RedisStreams(retention=, max_len=), with DEFAULT_RETENTION at seven days and no count cap by default. Both are checked at construction.
  • The append script trims exactly, in the same call as the append. A stage at or above max_len is refused where it's staged.
  • A replay that reaches a trimmed range fails its task with StreamIntegrityError, which the transport no longer wraps as a storage error. An outside cursor below the window is refused with StreamCursorError.
  • Tests for the bounds, the default, the loud replay failure, the age trim and the batch the window can't hold.

Part of AI-198 (epic AI-37).

Why?

Without a trim, a Redis log grows forever. The trim has no consumer floor, because keeping one would need a consumer registry in Redis. So the failure modes are made loud instead: a replay past the window fails rather than replaying something else. Trims are exact because Redis's approximate trim never touches a log shorter than one macro node.

How did you test it?

Link to a test plan if any -

  • Unit Tests
  • Staging
  • End to End Tests

poe lint is clean. The external stream suite passes on the dev server. The streams suite with STREAMS_LIVE=redis passes against a channel server and a local Redis.

A key is trimmed to a seven-day window by default, with an optional count cap,
on every append the provider makes. A replay that reaches a trimmed range fails
its task loudly, and an outside cursor below the window is refused.
The cases check the constructor's bounds, the seven-day default, a replay and a
read past the window, the age trim, and a batch the window cannot hold.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Changelog entry rides another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant