Skip to content

Gen migrate add varchar(N), blob(N) default, collates#279

Open
jongleb wants to merge 1 commit into
ygrek:masterfrom
jongleb:add-missing-revert-migrations
Open

Gen migrate add varchar(N), blob(N) default, collates#279
jongleb wants to merge 1 commit into
ygrek:masterfrom
jongleb:add-missing-revert-migrations

Conversation

@jongleb
Copy link
Copy Markdown
Collaborator

@jongleb jongleb commented May 11, 2026

Description

This PR adds proper support for VARCHAR(N), CHAR(N), VARCHAR2, BLOB(N), VARBINARY(N), TINYTEXT/MEDIUMTEXT/LONGTEXT (and their blob variants), and DECIMAL(p) in gen_migrations.

Previously these types were normalized into plain TEXT/BLOB/DECIMAL, so inverse migrations lost the original type flavor and length information.

Inverse migrations now also preserve COLLATE and DEFAULT clauses when reconstructing columns from the stored schema. Before this change, those attributes were silently dropped.

For now, DEFAULT values are stored as raw SQL fragments and replayed verbatim in inverse migrations. This is temporary. Once DEFAULT expressions are typechecked against the column type, they can be represented as structured values instead of opaque SQL text.

@jongleb jongleb force-pushed the add-missing-revert-migrations branch from b14a9c0 to 096f4c3 Compare May 11, 2026 17:24
@jongleb jongleb changed the title gen_migrate: varchar(N), blob(N) default, collates Gen migrate add varchar(N), blob(N) default, collates May 11, 2026
@jongleb jongleb requested review from Copilot and ygrek and removed request for Copilot May 11, 2026 17:25
@jongleb jongleb self-assigned this May 11, 2026
@jongleb jongleb marked this pull request as ready for review May 11, 2026 17:25
Comment thread lib/parser_state.ml Outdated
Comment on lines +14 to +26

let current_lexbuf : Lexing.lexbuf option ref = ref None

let with_lexbuf lexbuf f =
let saved = !current_lexbuf in
current_lexbuf := Some lexbuf;
Fun.protect f ~finally:(fun () -> current_lexbuf := saved)

let extract_source (start_, end_) =
Option.map (fun lexbuf ->
let offset = start_ - lexbuf.Lexing.lex_abs_pos in
String.trim (Bytes.sub_string lexbuf.Lexing.lex_buffer offset (end_ - start_))
) !current_lexbuf
Copy link
Copy Markdown
Collaborator Author

@jongleb jongleb May 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue even isn't that the default isn't unified. It's that it's any expression. This means we need expr->string, which fully restores the expression as an SQL string. And that requires a separate PR. So, we temporarily capture it as a string. Hence the hack in the parser state.

Comment thread lib/sql_parser.mly
Comment on lines +443 to +448
let pos = ($startofs(def), $endofs(def)) in
Some (Alter_action_attr.Default {
expr = make_located ~value:def ~pos;
sql = Parser_state.extract_source pos;
})
}
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A temporary solution is to keep the DEFAULT expression as-is until its type is unified with the column type.

@jongleb jongleb force-pushed the add-missing-revert-migrations branch 2 times, most recently from 7e385e0 to 02836d0 Compare May 12, 2026 13:44
@jongleb jongleb force-pushed the add-missing-revert-migrations branch 4 times, most recently from 305476e to 89149e5 Compare May 20, 2026 10:06
@ygrek
Copy link
Copy Markdown
Owner

ygrek commented May 20, 2026

i think it is ok to keep column attributes as text for the purpose of migration gen (use structured repr for other uses like typechecking etc), because it could be some custom database specific attributes that we don't parse and it seems better to do something reasonable (pass them verbatim) than just fail

@jongleb jongleb force-pushed the add-missing-revert-migrations branch from 89149e5 to 7cb2a71 Compare May 21, 2026 06:02
@jongleb jongleb force-pushed the add-missing-revert-migrations branch from 7cb2a71 to 3ff5779 Compare May 21, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants