Skip to content

Validate Postgres ClickPipe numeric minimums before API dispatch #1007

Description

@sdairs

Postgres ClickPipe creation accepts explicit zero values for five settings whose API schema requires positive minimums. The CLI should reject these locally with a flag-specific usage error before dispatching the request.

Reproduced with installed CLI 0.5.0, main df81e46cfec0bb58ea981d17cc6fd2de24442e7d.

CLI flag API field Minimum
--sync-interval-seconds syncIntervalSeconds 1
--pull-batch-size pullBatchSize 1
--initial-load-parallelism initialLoadParallelism 1
--snapshot-rows-per-partition snapshotNumRowsPerPartition 1000
--snapshot-parallel-tables snapshotNumberOfParallelTables 1

The bounds are explicit in the checked-in ClickPipePostgresPipeSettings API schema. The Postgres flags use Option<i64> without range parsers, and the builder forwards supplied values into settings.

A safe reproduction using only synthetic credentials and a loopback discard port:

clickhousectl cloud clickpipe create postgres \
  00000000-0000-0000-0000-000000000001 \
  --name qa-unused --host example.invalid --pg-database postgres \
  --username qa --password qa-placeholder \
  --table-mapping public.items:qa_items \
  --initial-load-parallelism 0 \
  --org-id 00000000-0000-0000-0000-000000000000 \
  --api-key qa-placeholder --api-secret qa-placeholder \
  --url http://127.0.0.1:9 --json

Actual: exit 1, structured transport error for the ClickPipe create endpoint. Replace the numeric flag with each of the other four flags set to zero: each likewise reaches transport instead of local validation. The loopback URL deliberately avoids cloud creation; this is not a claim that the deployed API accepts invalid settings. The hidden release --url override remains accepted for this test.

Expected: exit 2 before API dispatch, identifying the invalid flag and allowed minimum.

Acceptance:

  • Enforce each documented minimum on explicit values and expose the bound in actionable diagnostics/help.
  • Cover zero/negative values, 999 versus 1000 for partition rows, and each valid minimum. Preserve omitted numeric settings as omitted.
  • Verify rejection without network dispatch and valid request shapes using appropriate parser/builder/subprocess tests; do not pin help wording.

Related closed #565 introduced these settings; #449 covered different Postgres input constraints. Open #824 concerns applying whole-pipe PATCH settings, not local Postgres create validation. Searched all 460 existing open/closed issues before filing; no duplicate was found. No release milestone or blocker designation is requested.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions