Skip to content

Add clickhouse-native backend support#48

Open
tycooon wants to merge 3 commits intomainfrom
feat/clickhouse-backends
Open

Add clickhouse-native backend support#48
tycooon wants to merge 3 commits intomainfrom
feat/clickhouse-backends

Conversation

@tycooon
Copy link
Copy Markdown
Member

@tycooon tycooon commented Apr 24, 2026

Summary

UmbrellioUtils::ClickHouse becomes a polymorphic facade picking between two backends based on UmbrellioUtils.config.clickhouse_backend:

  • :legacy (default) — routes through the click_house HTTP gem, same behavior as before.
  • :native — routes through the clickhouse-native TCP gem.

Default stays :legacy, so existing consumers keep working unchanged.

Design

  • Backends::Base owns shared logic (from, count, db_name, DDL helpers, parse_value, pg_table_connection, populate_temp_table!, with_temp_table, sql_for, ch_dataset?, log_errors). Each concrete adapter implements the low-level ops (execute/query/query_value/query_each/insert/describe_table/tables/server_version/admin_execute/config/logger) plus a SERVER_ERROR class for error logging.
  • Backends::Legacy preserves the historical HTTP-driver behavior (memoized ClickHouse::Connection, per-host DNS resolution, Misc::StrictHash-wrapped JSON responses).
  • Backends::Native wraps ClickhouseNative::Pool, built from ::ClickHouse.config. create_database/drop_database route through admin_execute, a one-shot client against the always-present default db — unlike HTTP, the native TCP protocol needs a reachable database to issue DDL, so this unblocks rake ch:create on fresh environments.
  • New gem-level config UmbrellioUtils.config.clickhouse_native_settings lets native consumers set pool-wide session settings (e.g. allow_experimental_analyzer).

::ClickHouse.config shim

Native consumers without the old click_house gem don't get ::ClickHouse.config for free. A small shim at lib/umbrellio_utils/click_house/config.rb provides it (reading from Rails.application.config_for(:clickhouse)), gated by unless defined?(ClickHouse::Connection) so the legacy gem always wins when both are loaded. The shim is loaded eagerly from UmbrellioUtils.configure when clickhouse_backend == :native, so other gems that access ::ClickHouse.config directly (e.g. umbrellio-sequel-plugins' ch:create task) keep working.

Rake task

ch:connect now reads UmbrellioUtils::ClickHouse.config (which resolves via the active backend), so the task is backend-agnostic. Projects that locally overrode ch:connect to work around the old gem-binding can drop their override.

Ruby compatibility

clickhouse-native requires Ruby >= 3.3. Added it as a dev dep gated by install_if so CI still bundles on 3.1/3.2 (we skip the native backend spec context when the gem isn't installed).

Version

1.12.1 → 1.13.0 (new feature, backward compatible).

Test plan

  • All 154 existing specs pass (legacy backend behavior preserved).
  • New spec/umbrellio_utils/click_house/backend_dispatch_spec.rb covers legacy+native dispatch and unknown-backend errors.
  • Rubocop clean.
  • End-to-end verified against profile-service (110 CH-tagged specs pass on :native, rake ch:create/ch:migrate/ch:connect all work).

🤖 Generated with Claude Code

tycooon and others added 3 commits April 24, 2026 13:47
UmbrellioUtils::ClickHouse becomes a polymorphic facade with two
adapters: Backends::Legacy (existing `click_house` HTTP gem) and
Backends::Native (new `clickhouse-native` TCP gem). Consumers pick
the backend via `UmbrellioUtils.config.clickhouse_backend` — default
stays `:legacy` so existing users are unaffected.

Shared logic (from, count, DDL helpers, parse_value, temp-table
plumbing) lives on Backends::Base; each adapter provides the low-level
ops (execute/query/insert/describe_table/tables/server_version/
admin_execute) and a SERVER_ERROR class. DDL that creates/drops the
configured database routes through admin_execute so the native adapter
can open a one-shot client against the always-present `default` db
(the native TCP protocol needs a reachable db, unlike HTTP).

Native consumers also get a `::ClickHouse.config` shim (loaded eagerly
from `configure` when `:native` is set) so external code that reads
`::ClickHouse.config` directly — e.g. umbrellio-sequel-plugins' ch:create
rake task — keeps working without the old click_house gem.

ch:connect rake task is now backend-agnostic via
UmbrellioUtils::ClickHouse.config.

Version bump 1.12.1 → 1.13.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Allows consumers to inject a custom logger (e.g. a SemanticLogger
named channel) into the native ClickHouse pool, instead of the default
Rails.logger fallback.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Promotes the test-env-aware ON CLUSTER suppression that consumers
(profile-service, unetsafe) were duplicating in their local CH
modules. Adds:

- UmbrellioUtils.config.clickhouse_cluster (default "click_cluster",
  set to nil/blank to skip the clause everywhere).
- Backends::Base#on_cluster(sync:) — returns the ON CLUSTER clause,
  or "" when the cluster name is blank or `Rails.env.test?`. The
  cluster *name* is still used by callers like Distributed engine
  declarations regardless of this clause (each ON CLUSTER op blocks
  for hundreds of ms on a single-node CH waiting for replicas that
  don't exist).
- truncate_table! / drop_table! / optimize_table! and
  Migrations#create_distributed_table! now use the helper instead of
  hardcoding `ON CLUSTER click_cluster`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant