feat: add null_absent_columns() for cross-version schema tolerance - #398
Merged
Conversation
Apps add columns between releases. A query written against a newer store names a column an older one lacks, SQLite fails the whole statement with "no such column", and the artifact reports nothing rather than the rows it could have returned. One missing column costs every row. null_absent_columns(path, query) substitutes NULL for references the database does not have. It keeps the column in place and keeps its name, because artifacts read rows by name and a bare NULL renames the output column. Ported verbatim from iLEAPP, where it took ten modules from 132 error lines to 1 across 21 corpora and recovered rows on 56 artifact/corpus pairs (PR #1912). No call sites in this repo yet: this makes the helper available, it does not change any artifact's behaviour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Apps add columns between releases. A query written against a newer store names a column an older one lacks, SQLite fails the whole statement with
no such column, and the artifact reports nothing instead of the rows it could have returned. One missing column costs every row, and it fails quietly: an artifact that returns nothing looks exactly like a feature the user never used.null_absent_columns(path, query)substitutesNULLfor column references the database does not have.Ported verbatim from iLEAPP, where it took ten modules from 132 error lines to 1 across 21 corpora and recovered rows on 56 artifact/corpus pairs (iLEAPP #1912).
Three properties that are not obvious, and each cost a debugging round
NULL AS <name>, not a bareNULL, where the reference is a select item in its own right. A bare NULL also drops the output column's name, and artifacts read rows by name, so the SQL starts succeeding and the artifact then dies onrecord['ZFOO']withIndexError: No item with that key. Inside an expression the enclosing alias already names the column, so a plain NULL is correct there.NULL AS NULL. A sentinel does not save it either, because a non-word sentinel still satisfies the(?<![\w.])lookbehind.It resolves table aliases from the FROM/JOIN clauses, only touches references it can attribute to exactly one table, and returns the query unchanged if the schema cannot be read.
Scope
No call sites in this repo yet. This makes the helper available; it does not change any artifact's behaviour. Wiring it into specific artifacts is a separate change that deserves its own before/after corpus sweep.
Verified the ported function behaves identically to iLEAPP's in this core: same 97 rows on a real ThreeBars.sqlite missing
ZLOWQUALITY, column name preserved, valueNone.Worth noting for context: these five
ilapfuncs.pyfiles are less synchronised than "shared core" suggests — 43 top-level functions are common to all five, with 7 to 28 unique to each. So this is a deliberate addition rather than a sync.check_claim_language.py,check_html_safety.pyandlint_changed.pypass. Lint reports one fewer warning than before, because the helper usesreand silenced a pre-existing unused-import warning.Co-Authored-By: Claude Opus 5 noreply@anthropic.com