Skip to content

feat(mysql): A2 engine split — 8.4 opt-in for fresh installs + datadir format guard (backend#723 PR-2) - #597

Merged
LukasWodka merged 7 commits into
developfrom
feat/723-a2-mysql-84-optin
Aug 6, 2026
Merged

feat(mysql): A2 engine split — 8.4 opt-in for fresh installs + datadir format guard (backend#723 PR-2)#597
LukasWodka merged 7 commits into
developfrom
feat/723-a2-mysql-84-optin

Conversation

@LukasWodka

@LukasWodka LukasWodka commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Hold released 2026-08-05: docker.io/tracebloc/mysql-client:8.4 is live — client-runtime#264 merged, its publish workflow ran green on develop (multi-arch index verified: linux/amd64 + linux/arm64, MYSQL_MAJOR=8.4 on both platform configs). Rebased over #594/#604 (chart 1.9.15).

What (backend#723 execution plan · PR-2 — decision A2, Lukas 2026-08-05)

The fresh-vs-existing engine split, implemented explicitly (no lookup()):

Chart — mysql-format-guard init container (runs on every substrate)

Fails fast and loud when the engine major and the datadir format disagree, instead of letting mysqld CrashLoop cryptically:

  • 8.4 image over a 5.7-format datadir (no mysql.ibd) → refused: staged 5.7 → 8.0 → 8.4 only, with the runbook pointer.
  • 5.7 image over an 8.x-format datadir (mysql.ibd present) → refused: one-way conversion, points back at the 8.4 values or the pre-migration backup.
  • Fresh/empty datadir, the 8.0 transit hop, and unknown/custom digest pins → guard stands down (it's a backstop against accidental flips, not a gate).

tracebloc.mysqlEngineMajor derives the expected engine mirroring tracebloc.image's digest-wins precedence; the 5.7 digest literal in the helper is CI-pinned to the values default (re-pinning one without the other fails mysql_test.yaml). Non-root 999, read-only mount, no caps — same posture as the existing init container; on hostPath the chown bootstrap keeps running first.

Installer — _resolve_mysql_engine (bash installer only; PS1 untouched)

Engine choice for the generated values, most-specific first:

  1. TB_MYSQL_ENGINE=5.7|8.4 — explicit always wins (invalid values fail closed).
  2. Sticky: a previous 8.4 opt-in in this machine's values.yaml is kept (the file is regenerated on every run).
  3. Existing state pins 5.7: a found release (existing_id), or real datadir content under the legacy ($HOST_DATA_DIR/mysql) or per-release ($HOST_DATA_DIR/<ns>/mysql) layout — content, not -d, because _ensure_tracebloc_dirs pre-creates empty dirs.
  4. Fresh arm64 → 8.4 (native multi-arch instead of amd64 emulation — the cohort this ticket exists for). Fresh amd64 stays 5.7 for now: let 8.4 soak on arm64 + dev first, then flip all-fresh→8.4 as a one-line follow-up.

Opted-in installs get images.mysqlClient: {tag: "8.4", digest: ""} in the generated values — explicit tag, empty digest, so the chart's 5.7 reproducibility pin keeps applying to everyone else.

Test plan

  • helm-unittest 330/330 (+5: guard default-armed 5.7 — doubles as the digest↔helper sync pin; 8.4 opt-in image+guard; 8.0 transit; custom-digest → unknown; hostPath init order).
  • bats 92/92 (+13: 9 resolver units — explicit/invalid/sticky/existing-release/legacy-datadir/per-release-datadir/fresh-arm64/fresh-amd64; 4 full-flow — generated values with/without the block, arch-pinned for determinism).
  • shellcheck clean; bash -n clean.
  • Render diff vs develop (default values): exactly the guard container + chart-version labels — no other movement.

Sequencing

  1. tracebloc/client-runtime#264 (PR-1) merges → :8.4 publishes + self-verifies.
  2. Un-draft + merge this → rides the next chart release (existing fleets see only an inert guard).
  3. Fresh arm64 installs land on 8.4; Path-1/Path-2 fleet migration proceeds per the plan on backend#723 (fleet classification: @LukasWodka).

Chart 1.9.12 → 1.9.13 — same race as client#490/#594; loser rebases the version line.

Epic: backend#1151 (@saqlainsyed007) · plan + A2 decision record: backend#723 · PR-1: tracebloc/client-runtime#264

🤖 Generated with Claude Code


Note

Medium Risk
Touches MySQL startup and persistent datadir compatibility—wrong engine choice could block MySQL until values are corrected, though guards and installer heuristics aim to prevent that on existing edges.

Overview
MySQL engine split (backend#723, decision A2) — existing fleets stay on the pinned 5.7 image; fresh installs can run 8.4 without silent engine/datadir mismatches.

The chart adds a mysql-format-guard init container on every MySQL pod. It reads the expected engine from new helper tracebloc.mysqlEngineMajor (digest/tag precedence aligned with image pinning) and refuses startup when 8.4 would open a 5.7 datadir (no mysql.ibd) or 5.7 would open an 8.x datadir (mysql.ibd present), with runbook-oriented errors instead of mysqld CrashLoops. Empty datadirs, the 8.0 upgrade hop, and unknown/custom digest pins leave the guard inactive. On hostPath, the existing chown init still runs first, then the guard (read-only mount, non-root).

The bash installer gains _resolve_mysql_engine: TB_MYSQL_ENGINE (auto / 5.7 / 8.4), sticky 8.4 from prior values.yaml, and 5.7 when an existing release or real host datadir content is detected (fail-closed on unlistable dirs). auto selects 8.4 only for fresh arm64 installs and 5.7 otherwise; resolved 8.4 writes images.mysqlClient: { tag: "8.4", digest: "" } into generated values. Chart 1.9.21, expanded helm unittest / bats coverage, and values.yaml opt-in notes complete the change.

Reviewed by Cursor Bugbot for commit c28104b. Bugbot is set up for automated code reviews on this repo. Configure here.

@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

1 similar comment
@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

@LukasWodka
LukasWodka force-pushed the feat/723-a2-mysql-84-optin branch from bc25e34 to 1e4e735 Compare August 5, 2026 05:35
Comment thread scripts/lib/install-client-helm.sh
@LukasWodka

Copy link
Copy Markdown
Contributor Author

Bugbot's fail-open finding is fixed: the engine resolver now fails closed on an unlistable mysql datadir — the uid-999 --reuse-data case reads as content, keeping the reuse on 5.7 instead of wedging it behind the format guard (mirrors _leftover_data_dirs' stance for the same ownership case; chmod-000 regression test added). Also rebased over #593/#527/#525: manifest regenerated, and the new bats tests carry #527's || return 1 enforcement idiom. 95/95 bats, 330/330 chart tests, all static gates green.

bugbot run

@LukasWodka
LukasWodka force-pushed the feat/723-a2-mysql-84-optin branch from 1e4e735 to 8fc325b Compare August 5, 2026 07:13
@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 675dbff. Configure here.

LukasWodka and others added 4 commits August 5, 2026 12:10
…r format guard (backend#723 PR-2)

Chart: mysql-format-guard init container fails fast (with an actionable
message) when the engine major and datadir format disagree — 8.4 over a
5.7-format datadir and 5.7 over an 8.x one are both refused before mysqld
CrashLoops; the 8.0 transit hop and custom digest pins stand down.
tracebloc.mysqlEngineMajor derives the expected engine (digest-wins,
mirroring tracebloc.image); the 5.7 digest literal is CI-pinned to the
values default. Default render changes by exactly the guard.

Installer (A2, decision 2026-08-05): _resolve_mysql_engine picks the engine
for the generated values — explicit TB_MYSQL_ENGINE wins; a previous 8.4
opt-in is sticky; any existing release or real datadir content pins 5.7;
only a fresh arm64 install auto-selects 8.4 (native multi-arch instead of
amd64 emulation). amd64 fresh installs stay 5.7 for now (soak first).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ine resolver (Bugbot)

The auto branch treated a failed ls -A as an empty datadir — on arm64,
--reuse-data commonly leaves a uid-999 mysql dir the host user cannot
list, so the resolver opted the reuse into 8.4 and the format guard then
(correctly) refused the 5.7 datadir: the reuse path never came up. An
unlistable dir now counts as content (mirrors _leftover_data_dirs' fail-
closed stance for the same ownership case), with a chmod-000 regression
test. Rebased over #593/#527/#525 (manifest regenerated; my bats
negations now carry the #527 '|| return 1' enforcement idiom).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…hygiene guard

bats-hygiene's scanner requires every standalone bracket assertion in an
@test body to end in '|| return 1'; the resolver tests added on this
branch predated rebasing onto that guard.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
git hoisted the shared closing brace out of the conflict region during the
rebase onto #604; the file then died at parse (1 of 88 tests ran).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

@LukasWodka
LukasWodka force-pushed the feat/723-a2-mysql-84-optin branch from 675dbff to de5371e Compare August 5, 2026 10:13
@LukasWodka
LukasWodka marked this pull request as ready for review August 5, 2026 10:13
@LukasWodka
LukasWodka requested a review from saadqbal as a code owner August 5, 2026 10:13

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit de5371e. Configure here.

Comment thread client/templates/mysql-deployment.yaml Outdated
@LukasWodka

Copy link
Copy Markdown
Contributor Author

Real catch — a semantic rebase conflict with #604: the guard's busybox include predated the mirror re-homing and kept a hardcoded docker.io, which would ImagePullBackOff the always-on guard on mirrored/air-gapped edges. Fixed with the same dig "imageRegistry" expression every other image uses, plus a pin test (global.imageRegistry set → guard image re-homed). Chart tests green.

bugbot run

Semantic rebase conflict with #604: every other image include gained the
mirror dig while the guard (written pre-#604, merged clean textually)
kept a hardcoded docker.io — on mirrored/air-gapped edges the always-on
guard alone would ImagePullBackOff and block mysql on exactly the fleets
#604 serves. Same dig expression now + a mirror re-home pin test.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@LukasWodka

Copy link
Copy Markdown
Contributor Author

bugbot run

Conflict was only client/Chart.yaml: this branch was cut at 1.9.15 while
develop has since moved to 1.9.20. Resolved to develop's version rather than
re-bumping, because v1.9.20 is NOT tagged - the release train's rule is that
one bump covers a whole release cycle, so a PR riding an already-bumped,
still-unreleased version is exactly the green case (backend#1563).

Had v1.9.20 been released, this PR would instead need 1.9.21, since its delta
touches client/* which is client's declared publish_paths - the train would
otherwise publish different bytes under a shipped tag.

Verified after merging, because develop also changed installer scripts in
a07f76b: every scripts/manifest.sha256 entry still matches its file. A stale
manifest here would fail the install path at runtime rather than in review.
My earlier merge resolution took develop's 1.9.20 verbatim, reasoning from the
release train's rule: v1.9.20 is untagged, and the train's version_preflight
only refuses when the version is ALREADY released, so one bump covers a whole
release cycle. That reasoning is correct for the train and wrong for this repo.

client/scripts/chart-version-guard.sh enforces a stricter rule for a
repo-specific reason: chart content reaches installs only via a NEW chart
version, because a Helm repo publishes on version change. An unbumped
template/values edit therefore either reaches nobody or overwrites an
already-published version. Both have happened here - the perIngestionTables
block shipped dark in PR #472, and ingestor-0.2.0.tgz was overwritten 5x
between 2026-05-20 and 2026-07-29.

This PR changes client/templates/** and client/values.yaml, so it needs its
own version rather than riding develop's. v1.9.21 is untagged.
@LukasWodka
LukasWodka merged commit c519e44 into develop Aug 6, 2026
45 of 47 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants