[zuzmoskala]

Summary

• CI and local tests still use in-memory HSQL with Flyway off, so a bad MySQL migration usually shows up first on mojito-dev. This PR adds a local rehearsal stack so you can prove a schema upgrade against real MySQL before that deploy. As per requirements, nothing here replaces the HSQL tests, and it is not wired into CI.
• New Compose file docker/docker-compose-mysql-smoke.yml runs MySQL 8.0.34 and the current Java 21 webapp (Dockerfile-bk21) together. The webapp is started the way a MySQL deploy is: Flyway on, ddl-auto=none, no HSQL init scripts. Data lives in docker/.data/db on the host, so docker compose down keeps the database for the next run. Existing docker-compose.yml is left alone (it still targets Java 8 / MySQL 5.7).
• A populated upgrade is docker/smoke/populated-upgrade.sh. It builds the commit that introduced migration N (default 66) for linux/amd64, boots that image on an empty database, and Flyway stops at N because N is the newest migration in that image. It then seeds one repository, one messages.properties asset, and one French translation. A second image, built from the current working tree, replaces only the webapp. FLYWAY_TARGET stays unset, so that boot applies N+1 through the latest migration in the tree to the populated database. The script records the repository id, the asset id, and the translated string, and requires the same values after the upgrade. Java and SQL migrations both run through Mojito’s own Flyway inside the webapp.
• After an upgrade to latest, docker/smoke/check.sh confirms the app is UP, the running image still matches the sources it was built from, flyway_schema_history reached the highest integer version in the tree, and repo-type create / list / update / view / delete work. That repo-type check passes only when the database includes V69, the migration that adds the table. CLI calls go through docker/smoke/mojito-local, which only talks to the smoke container at localhost:8080.
• A green check.sh must not mean “old image, new files on disk.” Flyway checksums migrations in the running jar against flyway_schema_history; it does not read V* sources from the working tree. Edit a migration and leave an old process on 8080, and Flyway stays quiet. docker/smoke/build-image.sh records the image id and a fingerprint of every V*.sql / V*.java; docker/smoke/lib.sh hashes those files; docker/smoke/check.sh compares the running container and the current tree to that record (docker/.data/webapp-build.txt, gitignored). Rebuild with build-image.sh after changing migrations; up --build alone does not update the record. The optional host-jar edit loop is refused by check.sh.
• How to run it (fresh migrate, populated upgrade from N, smoke checks, and demo data) is in docker/readme.md.

Changes by file

• .dockerignore — excludes docker/.data from the webapp image build. MySQL runs in a separate container and keeps its files on the host via bind-mount; those files must not be copied into the Mojito image.
• .gitignore — ignores docker/local-jars/, the host-built jars copied for the edit loop.
• docker/Dockerfile-bk21 — removes a copied Mac node / node_modules tree before Maven, and reads MOJITO_WEBAPP_JAR / MOJITO_CLI_JAR when the container starts. Unset, both stay the jars baked into the image.
• docker/docker-compose-mysql-smoke.yml — the smoke stack: persisted MySQL, production-like Flyway settings, TCP healthcheck so the webapp does not start while port 3306 is still down, FLYWAY_TARGET mapped to spring.flyway.target. The populated-upgrade recipe leaves that variable unset.
• docker/docker-compose-mysql-smoke.local-jars.yml — opt-in edit loop. It mounts host jars over the baked jars. check.sh refuses this mount, so it is not the trusted smoke run.
• docker/readme.md — runbook for the harness: fresh DB, populated upgrade from the commit that introduced N, smoke checks, and the integer-version caveat. N is a plain integer with no leading zero. Supported bases start at V66, the first migration introduced by a Java 21 commit.
• docker/smoke/build-image.sh — builds the webapp image and records its ID plus a fingerprint of every V*.sql / V*.java file, so a later check can tell a stale image from the current tree.
• docker/smoke/lib.sh — shared hashing helpers for that fingerprint (Linux sha256sum or macOS shasum).
• docker/smoke/check.sh — the pass/fail smoke: health, image/source freshness, Flyway version, and repo-type CRUD (including that delete actually removed the type).
• docker/smoke/mojito-local — runs the CLI inside the smoke webapp container and refuses any other host, so a laptop mojito pointed at a real server cannot be used by accident. Color codes are off, so repo-view prints a plain repository id.
• docker/smoke/populated-upgrade.sh — the populated upgrade. It checks N before asking to delete docker/.data/db, accepts yes or y in any case with surrounding spaces, builds the historical image, seeds, boots the current working tree on the same database, and compares the repository id, asset id, and translation.
• docker/smoke/fixtures/populated-upgrade/ — the English source and French translation that seed uses.
• docker/smoke/stage-local-jars.sh — copies the host-built webapp and CLI jars onto stable names for the edit-loop mount.

Test evidence

Verified locally on this branch (flyway-mysql-smoke-harness) with Colima. Sections A, B, and D were run on 2026-10-05. Section C was run on 2026-10-19. CLI calls go through docker/smoke/mojito-local against the smoke stack. Image builds use docker/smoke/build-image.sh except for the historical image in section C, which is built by docker/smoke/populated-upgrade.sh. Stack: docker/docker-compose-mysql-smoke.yml.

A — Fresh database (V1 → latest)

• Empty data dir, up -d with no FLYWAY_TARGET, docker/smoke/check.sh passes at Flyway 69. Health UP; repo-type create / list / update / view / delete; delete removes only the type that run created.

$ docker compose -f docker/docker-compose-mysql-smoke.yml down
$ rm -rf docker/.data/db && mkdir -p docker/.data/db
$ docker compose -f docker/docker-compose-mysql-smoke.yml up -d
$ docker/smoke/check.sh
Waiting for http://127.0.0.1:8080/actuator/health
{"status":"UP"}
Running webapp image matches the SQL and Java migrations from its build.
Running webapp SQL migrations match the working tree.
flyway_schema_history max version: 69 (highest migration: 69)
created --> repo type id: 1
Description --> local smoke check updated
No repo types found
Smoke checks passed (Flyway version 69).

• Logbook versions 1–69, all success = 1, one boot (installed_on 10:29:11–13). Java migrations recorded as JDBC: V9 Compute Word Count, V56 TUCVAddAssetIdUpdater. V69 Add repo type. Tables repo_type, repo_type_integrity_checker, and column repository.repo_type_id exist.

$ docker compose -f docker/docker-compose-mysql-smoke.yml exec db \
    mysql -umojito -p<PASSWORD> mojito -e "…"

version 1   Initial Setup           SQL   2026-10-05 10:29:11
version 9   Compute Word Count      JDBC  2026-10-05 10:29:11
version 56  TUCVAddAssetIdUpdater   JDBC  2026-10-05 10:29:13
version 66  user locales            SQL   2026-10-05 10:29:13
version 67  Add bn BD               SQL   2026-10-05 10:29:13
version 68  Add ai review proto     SQL   2026-10-05 10:29:13
version 69  Add repo type           SQL   2026-10-05 10:29:13

TABLE_NAME: repo_type, repo_type_integrity_checker
repository.repo_type_id: bigint

(Full 1–69 captured locally; every version succeeded.)

B — Persistence across compose down

• demo-create -n Demo inserts repository id 1. down removes containers and leaves docker/.data/db on disk. Next up -d (no rebuild, no FLYWAY_TARGET): still Flyway 69, repo-view -n Demo still id 1.

$ docker/smoke/mojito-local demo-create -n Demo
Create demo repository: Demo
Demo repository is ready!

$ … mysql -umojito -p<PASSWORD> mojito -e "SELECT id, name FROM repository;"
|  1 | Demo |

$ docker compose -f docker/docker-compose-mysql-smoke.yml down
✔ Container mojito-mysql-smoke-webapp-1 Removed
✔ Container mojito-mysql-smoke-db-1     Removed

$ ls -ld docker/.data/db
drwxr-xr-x  … docker/.data/db

$ docker compose -f docker/docker-compose-mysql-smoke.yml up -d
$ docker/smoke/check.sh
Smoke checks passed (Flyway version 69).

$ docker/smoke/mojito-local repo-view -n Demo
Repository id --> 1

max_version: 69
|  1 | Demo |

C — Populated upgrade from the commit that introduced V66

docker/smoke/populated-upgrade.sh builds that commit, migrates an empty database to V66, seeds a repository, then replaces only the webapp with an image of the current working tree. FLYWAY_TARGET stays unset. The second boot applies the later migrations to the populated database.

• First image was commit 214b2f9670c5ea9ca5969a5794dd55ee0a19fee7. Deleting docker/.data/db required typing y. The script continued to the seed only after that image's Flyway version was 66. push created one asset, id 1. The number after task: is that upload's background job id.

$ docker/smoke/populated-upgrade.sh
Checks passed for V66 (214b2f9670c5ea9ca5969a5794dd55ee0a19fee7).
Type yes or y to delete docker/.data/db: y
--> asset id: 1, task: 2

• After the current-tree image booted on the same database, the repository id was still 1, the asset id was still 1, and pull returned the imported French line. check.sh passed at Flyway 69, including repo-type create / list / update / view / delete. Compose down removed the containers and left docker/.data/db on disk.

/opt/smoke-pull/messages_fr-FR.properties:greeting=Bonjour de la mise à niveau du schéma
Repository 1, asset 1, and translation survived the upgrade.
{"status":"UP"}
Running webapp image matches the SQL and Java migrations from its build.
Running webapp SQL migrations match the working tree.
flyway_schema_history max version: 69 (highest migration: 69)
Description --> local smoke check updated
No repo types found
Smoke checks passed (Flyway version 69).
✔ Container mojito-mysql-smoke-webapp-1 Removed
✔ Container mojito-mysql-smoke-db-1     Removed
✔ Network mojito-mysql-smoke_default    Removed

D — Editing an already-applied migration (negative)

• Comment appended to V67 (not V1; Mojito has a special V1 checksum repair), image rebuilt, webapp recreated. Process Exited (1). Migration checksum mismatch for migration version 67. MySQL stayed up.

$ printf '\n-- smoke-harness checksum probe\n' >> webapp/src/main/resources/db/migration/V67__Add_bn_BD.sql
$ docker/smoke/build-image.sh
$ docker compose … up -d --no-deps --force-recreate webapp

STATUS: Exited (1)
Migration checksum mismatch for migration version 67
Validate failed: Migrations have failed validation

• Restored that file with git checkout --, rebuilt, recreated webapp. check.sh passes at 69. V67 no longer modified.

$ git checkout -- webapp/src/main/resources/db/migration/V67__Add_bn_BD.sql
Smoke checks passed (Flyway version 69).

[ehoogerbeets]

The MySQL 8 stack, the TCP healthcheck, and the image-versus-source check are the right shape for a local rehearsal, and I would leave this out of CI.

Inline comments on the image name, sort locale, seeding real rows before the upgrade, Mac node in Dockerfile-bk21, and bind-mounting recompiled jars with -v so a code change does not require a full image rebuild.

[ehoogerbeets]

This first boot is a database at N, and the app is allowed to exit because the image is current Mojito on an older schema. The upgrade path never writes the rows a real upgrade has to keep: a repository, an asset, text units, and imported translations.

An empty migrate accepts a NOT NULL column with no default, a table rebuild, or a type change that fails or drops data once those tables have rows. That is the failure we would hit going from one schema to the next on a populated mojito, and this recipe does not exercise it.

A closer rehearsal, on the same docker/.data/db:

1. Boot a build whose own latest migration is N (the revision that introduced V{N}), with no FLYWAY_TARGET. Code and schema match, so the app stays up. FLYWAY_TARGET on the current image cannot host this step when current code will not run on schema N. The readme already says that boot may exit.
2. Seed through docker/smoke/mojito-local: repo-create, push a small source file, import translations for those strings. The source file has to be visible in the container (docker compose cp, or a bind mount). Record the repo id and one translated string. Skip repo-type-create when N is below V69; that table is not there yet.
3. Leave the data directory in place. Recreate the webapp from the new image with FLYWAY_TARGET unset, so Flyway applies only N+1 through latest.
4. Confirm the repo, the asset, and the imported translation are still there (repo-view, pull or export). Then run check.sh, which covers the repo-type API on the upgraded schema.

check.sh only sees the final version, so it cannot tell this populated upgrade from a fresh migrate. The seed has to be checked before the second boot and again after.

[zuzmoskala]

This is now fixed. Script is in docker/smoke/populated-upgrade.sh
It:

1. Checks N, then asks you to confirm deletion of docker/.data/db. The answer is yes or y; capital letters and surrounding spaces are accepted. That directory holds the MySQL data and survives docker compose down. The first boot has to migrate an empty database to V{N}. Flyway records applied versions in this directory, so a leftover directory makes this run continue from the schema already stored there.
2. Builds the past commit that introduced V{N}. The script puts that commit in a temporary directory. Your checkout stays on the branch you already have. This image starts on an empty database, and Flyway stops at V{N} because that is the newest migration in that old Mojito.
3. Seeds a repository and checks the repository id, the messages.properties asset id, and the French translation.
4. Builds a second image from the files in your current working tree. This is the branch you already have checked out, not the old commit from step 2. Only the webapp container is replaced. MySQL and docker/.data/db stay, so this webapp opens the populated V{N} database and Flyway applies N+1 through the latest migration in this tree.
5. Checks that the same repository id, asset id, and translation are still there.
6. Runs docker/smoke/check.sh. That check creates a repository type, so it passes only when the upgraded database includes V69, the migration that adds that table.

(I documented the flow above in readme.md).

[zuzmoskala]

I tested it locally, ran docker/smoke/populated-upgrade.sh locally on Colima. The first image was 214b2f9, the commit that introduced V66. I confirmed deletion of docker/.data/db, seeded one repository, and then booted the current working tree against that same database.

Push reported one asset:

--> asset id: 1, task: 2

After the upgrade:

/opt/smoke-pull/messages_fr-FR.properties:greeting=Bonjour de la mise à niveau du schéma
Repository 1, asset 1, and translation survived the upgrade.
flyway_schema_history max version: 69 (highest migration: 69)
Smoke checks passed (Flyway version 69).

check.sh created, updated, and deleted a repository type. The closing list printed "No repo types found". Compose down removed the containers. docker/.data/db stayed on disk.

[ehoogerbeets]

Still one comment left to address

[zuzmoskala]

Still one comment left to address

Yep, this is work in progress, I'll let you know once done.