fix(warehouse-sources): hold imports for every converging rewrite

[posthog]

Problem

• A large warehouse table that needs more than one activity budget to repartition can never finish, so it stays coarsely partitioned and keeps meeting the same OOM or size trigger that flagged it.
• A rewrite checkpoint resumes only while the live Delta version is unchanged, and the schema's own merge moves it. The import hold that prevents that merge was gated behind a rollout flag targeting a single schema.
• Every other table therefore discarded its checkpoint each sync, re-streamed from row 0 with the same budget, and was abandoned terminally under a cooldown.
• The terminal skip also called capture_exception, so a handled outcome opened an error tracking issue next to the warehouse_repartition_skipped event that already records it.

Changes

Situation	Before	After
Staged swap	Import held	Import held (unchanged)
Live rewrite checkpoint	Held only for flag-targeted schemas	Held for every schema
Terminal skip	Skip event and error tracking issue	Skip event only

• A rewrite that runs out of budget now pauses that one schema's imports until it converges, so the version fence stays valid and the table finishes across runs.
• The hold stays self-limiting. held_at is restamped on every checkpoint write, so a rewrite that stops advancing releases the hold and the worst case is a stale table, never a stopped one.
• Mechanical: the data-warehouse-repartition-hold flag constant and its helper are removed, and comments that explained the gate are updated. The flag row can be deleted after this deploys.

Note

This widens who can be held. Both holds follow from a repartition the schema already opted into, and holding is the cheaper trade: the alternative is not a fresher table, it is a rewrite that burns a full activity budget per run and converges on nothing.

How did you test this code?

• pytest products/warehouse_sources/backend/temporal/data_imports/workflow_activities/tests/test_import_data_sync.py -k repartition and .../pipelines/core/test_repartition.py pass locally.
• The database-backed test_repartition_controller.py was not run: this sandbox has no Postgres, so it was only collected. CI runs it.
• New coverage, and the regression each case catches:
  • test_an_in_flight_repartition_holds_the_import gains a rewrite_checkpoint case: it fails if the rewrite branch is re-gated or dropped, which is the exact defect fixed here. Extended rather than added, because the swap case already covers this function.
  • test_a_terminal_rewrite_error_clears_pending gains one assertion: a terminal skip must not report an exception.
• No manual or production verification was performed.

Automatic notifications

• Publish to changelog?

Docs update

None.

🤖 Agent context

Autonomy: Fully autonomous

• Written by Claude Opus 5 in a PostHog cloud task from a Self-driving inbox report.
• Skills invoked: /writing-tests, /writing-simplified-technical-english, plus the repo comment and PR-description conventions.
• The report described the rewrite hold as missing. It exists on master but sits behind a per-schema rollout flag, so the effect was the same for every table the flag did not name. Production telemetry confirmed it: only flag-targeted schemas ever recorded a rewrite_converging hold. The fix therefore removes the gate rather than adding a hold.
• Alternative considered and rejected: widen the flag's release conditions. That fixes one table by hand and leaves the next one to fail the same way.
• Blast radius was checked against telemetry before removing the gate. Only a few schemas a day hold a rewrite checkpoint at all.
• No duplicate: gh pr list --state open --search repartition found no PR covering this. PR chore(warehouse): remove the always-on auto-coarsen rollout flag #95006 removes a different always-on warehouse rollout flag.
• Public artifact: no customer names, row counts, or session material reached the diff or this description. Test fixtures use invented values.

---

Created with PostHog Desktop from this inbox report.

[trunk-io]

Merging to master in this repository is managed by Trunk.

• To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

[posthog]

🦔 PostHog Review reviewed this pull request

Found 1 must fix, 3 should fix, 1 consider.

Published 5 findings (view the review).

Resolved comments: 4 fixed

[github-actions]

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

[stamphog]

Not approved — escalated to a human reviewer.

Re-add the stamphog label to request another review once you have addressed this.

This removes a rollout-flag gate so that every schema (not just flag-targeted ones) now pauses its import while a multi-run repartition rewrite converges — a production behavior change to the warehouse data-import path with no independent review, and the bot author carries no familiarity signal to substitute for one.

• Widens the import-hold behavior from flag-targeted schemas to all schemas with a converging rewrite checkpoint, with zero reviews on the PR.
• Author is posthog[bot] (machine user via self-driving inbox task), not on the owning team, and familiarity carries no signal for this author type — no independent assurance covers this risky-territory change.

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✓	no deny categories matched
size	✓	52L, 4F substantive, 87L/6F incl. docs/generated/snapshots — within ceiling
tier	✓	T1-agent / T1c-medium (87L, 6F, single-area, fix)
stamphog 2.0.0b4		.stamphog/policy.yml @ 55c523b · reviewed head 55c523b

[posthog]

PostHog Review alpha 🦔 If you find any issues helpful - please reply "valid", "invalid", etc., for evaluation purposes 🙏

[posthog]

PostHog Review

Found 1 must fix, 3 should fix, 1 consider.

Other findings (outside the changed lines)

Valid issues on this PR's files that sit on lines GitHub won't let us comment on inline.

Terminal budget failures lose stable observability

Priority: consider | File: products/warehouse_sources/backend/temporal/data_imports/workflow_activities/repartition_table.py:496-512 | Category: best_practice

Why we think it's a valid issue

• Checked: the terminal branch at repartition_table.py:496-513, the sibling telemetry paths in the same file (_capture_stood_down at :602-624, _handle_failure at :841-877, _give_up at :735-754), the metric label space, and the raise site of RepartitionTooLargeForBudgetError at pipelines/core/repartition.py:1347.
• Found: the branch catches two outcomes of unequal severity. RepartitionUnpartitionableError is benign, so dropping its report is correct. RepartitionTooLargeForBudgetError marks a table the controller abandons for good, and this PR's own docstring edit at repartition.py:122-134 narrows its meaning further: with the hold now always on, reaching it means the temp is gone or the hold lapsed, so it is an anomaly, not a designed outcome.
• Found: the surviving signals lack the stable fields every sibling path carries. props.update({"trigger_reason": ..., "reason": str(e)}) at :508 writes the raw message, which for the budget error embeds rows_written, the live row count, and schema_id (repartition.py:1347-1351), so the value differs on each occurrence. _capture_stood_down sets "terminal": False at :618, and both failure paths set error_type at :745 and :841. The terminal skip sets neither. The branch also emits no log line.
• Found: the metric label is shared. outcome="skipped" at :510 is the same label the completed path emits at :575 for the benign live_unreadable and no_delta_table results (repartition.py:1261, :1283), so the counter cannot separate an abandoned table from a table that was simply unreadable this run.
• Found: the removal contradicts a rule the file states twice. repartition_table.py:750-753 says the give-up captures "so a table the controller has abandoned surfaces as an issue instead of only a metric", and calls itself "the only terminal path that did not already report to error tracking". After this diff that sentence is false, and repartition.py:216 repeats the same principle for RepartitionAttemptsExhausted.
• Impact: a table that can never converge is now visible only through one analytics event whose grouping key changes per occurrence, plus a counter label shared with benign skips. The table stays on the coarse layout that triggers the OOM, and the cooldown re-arms it at most daily, so the condition persists while nothing distinct reports it.
• Priority: lowered to consider. The event and the counter still fire, and an operator can match the reason text, so "cannot alert" overstates the loss. The path is also rarer after the hold fix. The gap is real and the fix is two props, but it costs telemetry shape, not correctness or user data.

Issue description

This deletion suppresses both terminal exception types. RepartitionTooLargeForBudgetError means a budget-exhausted checkpoint became unusable. The remaining metric combines this failure with benign skips. The event stores only a changing message with row counts and schema data. Operators cannot reliably group or alert on failed convergence.

Suggested fix

Add stable fields such as terminal=True, error_type=type(e).__name__, and a bounded reason_code. Alert on the budget-error code before removing its exception report.