Skip to content

fix: suppress bool->int8 false-positive column type change from TABLE_MAP metadata - #4726

Open
shockdav wants to merge 4 commits into
PeerDB-io:mainfrom
shockdav:fix/tinyint1-bool-int8-tablemap-false-positive
Open

fix: suppress bool->int8 false-positive column type change from TABLE_MAP metadata#4726
shockdav wants to merge 4 commits into
PeerDB-io:mainfrom
shockdav:fix/tinyint1-bool-int8-tablemap-false-positive

Conversation

@shockdav

Copy link
Copy Markdown
Contributor

MySQL/MariaDB binlog TABLE_MAP metadata cannot express the TINYINT(1) display width. A column that the snapshot typed as bool therefore always arrives on the wire as int8, and shouldReportColumnTypeChange reports it as a column type change once per row-event, forever.

On a busy table this is unbounded log growth. In our deployment (MariaDB 10.x → ClickHouse, stable-v0.37.0) a single tinyint(1) column on the hottest table produced ~34 GB of this one WARN line — about 83% of the flow-worker's total log volume — before it filled the host's disk and stalled the mirror:

{"level":"WARN","msg":"column type change detected from TABLE_MAP_EVENT, not propagating","table":"tradersync.trades","column":"symboldata","from":"bool","to":"int8"}

The source column was, and still is, tinyint(1); nothing changed. The mirror column is Bool and the data is correct either way — the warning is a pure false positive.

This suppresses exactly the bool (schema) → int8 (wire) pairing, mirroring the existing MariaDB UUID/INET wire-kind special case. A real change — bool → anything other than int8, or any non-bool schema kind — is still reported.

@shockdav
shockdav requested a review from a team as a code owner August 25, 2026 14:38
@shockdav
shockdav requested a deployment to external-contributor August 25, 2026 14:38 — with GitHub Actions Waiting
@shockdav
shockdav requested a deployment to external-contributor August 25, 2026 14:38 — with GitHub Actions Waiting
@CLAassistant

CLAassistant commented Aug 25, 2026

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ ilidemi
❌ shockdav
You have signed the CLA already but the status is still pending? Let us recheck it.

@shockdav

Copy link
Copy Markdown
Contributor Author

Pushed a second commit covering the same failure mode for ENUM: binlog TABLE_MAP encodes ENUM under the generic STRING column type, so a snapshot-typed enum column always arrives as string on the wire and warns once per row-event forever (we measured 4 enum columns on one table producing ~90 warn lines/min). Same shape of suppression, scoped to exactly the enum->string pairing.

@shockdav
shockdav had a problem deploying to external-contributor August 25, 2026 17:46 — with GitHub Actions Failure
@shockdav
shockdav temporarily deployed to external-contributor August 25, 2026 17:46 — with GitHub Actions Inactive

@ilidemi ilidemi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution @shockdav! Would you be able to sign the CLA so we can merge?

@ilidemi
ilidemi temporarily deployed to external-contributor August 25, 2026 21:10 — with GitHub Actions Inactive
@ilidemi
ilidemi temporarily deployed to external-contributor August 25, 2026 21:10 — with GitHub Actions Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants