Skip to content

WIP insert multiple batches at once from s3/gcs to clickhouse - #4732

Draft
dtunikov wants to merge 1 commit into
bq/isolate-tables-flowfrom
bq/insert-multiple-batches-at-once
Draft

WIP insert multiple batches at once from s3/gcs to clickhouse#4732
dtunikov wants to merge 1 commit into
bq/isolate-tables-flowfrom
bq/insert-multiple-batches-at-once

Conversation

@dtunikov

@dtunikov dtunikov commented Aug 26, 2026

Copy link
Copy Markdown
Contributor
  • remove random() from s3 file names
  • use regex to insert multiple s3 files at once, e.g.:
    INSERT INTO ... SELECT ... FROM s3(['url1', 'url2'], 'access_key', 'secret_key', 'Avro')

@dtunikov dtunikov changed the title insert multiple batches at once from s3/gcs to clickhouse WIP insert multiple batches at once from s3/gcs to clickhouse Aug 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant