Repository navigation
Conversation
…efault A new hourly maintenance:retention job deletes old rows in bounded batches (5,000 rows, SKIP LOCKED, DB clock, a time budget per run) from deliverability_events, inbox threads and messages, tracking_events, sends and task_dead_letters. Every table that identifies recipients is disabled unless the operator sets a window (INROAD_RETENTION_*_DAYS). The window is a Privacy/Legal decision, not a code default. Each table has a minimum below which the worker refuses to start. Dead letters keep their previous 90-day purge, now configurable. What "delete" means per table: - Tracking events are rolled up into tracking_event_rollups in the same statement that deletes them. Reports now read the tracking_engagement view (raw + rolled-up rows), so historical counts survive. - Sends: only terminal rows, and never while an enrollment, guardrail window, inbox thread, deliverability event or CRM deal still depends on them. - Deliverability events: kept while they feed a running campaign's auto-pause window, and each workspace's newest complaint is kept. - Inbox: only whole conversations, never with a pending reply or an active snooze. The job runs in-process only. It deletes across workspaces, so the remote client deliberately does not implement it (invariant 73), and the census test asserts that. The tenancy scanner now treats workspace-scoped views as tenant tables. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e review's correctness fixes Performance audit: - The migration no longer locks sends for the whole deploy. The first migration holds only instant DDL, and each index is its own single-statement CREATE INDEX CONCURRENTLY file (asserted indisvalid; a static test enforces one statement per file). - Batches are scan-bounded, not delete-bounded: a candidate CTE over (age, id), guards written NOT EXISTS ... OFFSET 0 so they are probed per candidate, and a cursor persisted in retention_cursors. A 60s per-batch statement_timeout. - One sweeper per run via a session advisory lock. This also removes the cross-replica rollup/purge deadlock. - New non-partial index on inbox_pending_replies(thread_id), so thread deletes don't seq-scan. The redundant idx_tracking_events_send is dropped. Autovacuum reloptions, and a first-enable procedure in the docs. Correctness review: - The breaker-history guards now cover paused campaigns too, so a resumed campaign isn't auto-paused on inflated evidence. - A test fails if any new query reads raw tracking_events outside an explicit allowlist. Readers must use tracking_engagement. - A tracking hit that races a purge gets the documented 404 (ErrSendGone). - The docs cover what late replies, bounces and ARF complaints lose once their send is deleted, and list the recipient-data tables no window covers (a Privacy/Legal call). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft: merge after #242 (branching). Once that merges, this PR is rebased to:
tracking_engagement, with a test that rolls up an open and then evaluates a branch on it. Without this, an "opened within 60 days" branch would miss opens older than the retention window and silently send the wrong email.idx_inbox_threads_campaign_contact, with different columns.This covers the "[Scale] No retention on any high-volume table" Asana task.
What it does
An hourly
maintenance:retentionjob deletes old rows from:deliverability_eventstracking_events(rolled up intotracking_event_rollupsfirst, so reports survive)sendstask_dead_lettersEvery recipient-identifying table is OFF unless the operator sets a window. The windows are a Privacy/Legal decision, and PIPEDA or Quebec Law 25 may apply. The docs list the recipient-data tables no window covers.
Safety
CREATE INDEX CONCURRENTLYfile per index (assertedindisvalid). A static test enforces one statement per file.tracking_engagementview (raw plus rolled-up rows), so historical counts don't change. A test fails on any new rawtracking_eventsreader.Reviews
Verification
-tags=integration) and golangci-lint v2.12.2 pass.internal/platform/storagefails locally with Windows file locks. It isn't touched here.🤖 Generated with Claude Code