Skip to content

create --partitions=(tuple) kills process with 'Cannot parse expression of type Date' when backup scope contains tables with different PARTITION BY arity (including skipped system.* tables) #1547

Description

@Slach

Symptom

create --partitions=(202504,'2025-04-16') (tuple partition value, without db.table: prefix) kills the whole process (in server mode: the API server restarts) with:

WRN pkg/partition/partition.go:156 > partitionId function failed, using temporary table: partition values count (2) doesn't match components count (1)
FTL pkg/partition/partition.go:488 > partition.GetPartitionIdAndName error: getPartitionIdWithTempTable insert partition: code: 62, message: Cannot parse expression of type Date here: 202504,'2025-04-16'): While executing WaitForAsyncInsert

Reported against 2.8.0, reproduced on master (c4a5b62e), ClickHouse 26.8.2.7.

Reproduce

CREATE DATABASE vs;
CREATE TABLE vs.test_tuple (a Int64, b Int64, c Int64, d String, year_month UInt32, event_date Date)
ENGINE = MergeTree PARTITION BY (year_month, toString(event_date)) ORDER BY (a, b, c);
INSERT INTO vs.test_tuple VALUES (1,2,3,'p1',202504,'2025-04-16'),(5,6,7,'p3',202505,'2025-05-01');
-- any other table with single-component PARTITION BY in scope
CREATE TABLE vs.single (event_date Date, x UInt8) ENGINE=MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY x;
INSERT INTO vs.single VALUES ('2025-04-16',1);
clickhouse-backup create --tables='vs.*' "--partitions=(202504,'2025-04-16')" t1   # FTL as above
clickhouse-backup create --rbac         "--partitions=(202504,'2025-04-16')" t2   # FTL on system.metric_log (see below)
clickhouse-backup create --tables=vs.test_tuple "--partitions=(202504,'2025-04-16')" t3   # works

Root cause

partition.ConvertPartitionsToIdsMapAndNamesList applies a --partitions value without db.table: prefix to every table returned by GetTables, and calls log.Fatal() on the first table where GetPartitionIdAndName errors. The error itself is not about vs.test_tuple (its partition id is resolved fine via partitionId()), it comes from another table whose PARTITION BY has a different arity:

  1. getPartitionIdWithFunction correctly detects partition values count (2) doesn't match components count (1) and returns an error.
  2. GetPartitionIdAndName treats any error from the function path as "fall back to the temporary table", including this arity mismatch.
  3. getPartitionIdWithTempTable then does INSERT INTO __partition_id_<table>(event_date) VALUES (?,?) with two values into one column → code: 62 Cannot parse expression of type Date here: 202504,'2025-04-16').
  4. ConvertPartitionsToIdsMapAndNamesList turns that into log.Fatal().

Two contributing bugs:

  • Skipped tables are not excluded. GetTables returns tables with Skip=true (e.g. system.* from skip_tables), and ConvertPartitionsToIdsMapAndNamesList iterates them anyway. That is why create --rbac --partitions=... without --tables dies on system.metric_log (PARTITION BY toYYYYMM(event_date); there it fails even earlier with Max query size exceeded on the huge CREATE TABLE __partition_id_metric_log). Partition id resolution should never touch skipped tables.
  • Arity mismatch is treated as a fatal error instead of "value does not apply to this table". A tuple value with N components can never match a table whose PARTITION BY has M != N components; the function path already knows this. Falling back to the temp-table path (which only produces a confusing ClickHouse parse error) and then log.Fatal() makes a single --partitions value unusable in any backup scope that contains tables with different PARTITION BY shapes. The mismatch should be a warning and the table should get no partition filter entry for that value (same as today when the value simply matches no part), or at minimum a returned error instead of log.Fatal() that kills the API server.

Note: --partitions values with db.table: prefix are unaffected because tablePattern scoping is applied.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions