TCP T3: usage examples for the native client - #595
Draft
alex-clickhouse wants to merge 16 commits into
Draft
Conversation
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
August 30, 2026 09:24
ea60231 to
1ba7edc
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
August 30, 2026 18:59
1ba7edc to
f76c287
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
August 31, 2026 09:27
f76c287 to
502f20d
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
2 times, most recently
from
August 31, 2026 16:40
4e0629f to
dc0e282
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
August 31, 2026 17:23
dc0e282 to
ef198e1
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
September 1, 2026 08:23
ef198e1 to
b17539e
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
September 1, 2026 08:56
b17539e to
fc8592f
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
September 1, 2026 18:20
fc8592f to
134556e
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
September 2, 2026 09:04
134556e to
3c71212
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
September 2, 2026 11:07
3c71212 to
a12aea3
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
September 3, 2026 09:25
a12aea3 to
047623b
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
September 3, 2026 14:25
047623b to
753346e
Compare
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
September 4, 2026 09:02
753346e to
982eb6c
Compare
The HTTP examples move under examples/Http/, making room for a Tcp/ series alongside them. Declared namespaces are unchanged, since ExampleRunner matches ClickHouse.Driver.Examples exactly whatever folder a file sits in. AGENTS.md carries the checklist for adding an example, because RunAllExamples in Program.cs is hand-maintained: an example left out of it still compiles, still lists, and still runs under --filter, so nothing reports that CI never runs it. DateTimeHandling, AsyncInsert and QBitSimilaritySearch were in that state; they are registered now, and the three that need a cluster, Cloud credentials or a JWT are listed as deliberate omissions. QBitSimilaritySearch read L2DistanceTransposed with GetFloat, which throws because the server returns Float64. The README's filter section described matching on the file name; the runner matches the class name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ExampleConfig assembles the connection string from CLICKHOUSE_HOST, CLICKHOUSE_HTTP_PORT, CLICKHOUSE_TCP_PORT, CLICKHOUSE_USER, CLICKHOUSE_PASSWORD and CLICKHOUSE_DATABASE, each falling back to what a stock server container exposes on localhost, so the suite runs with nothing set and points somewhere else without editing 46 files. CLICKHOUSE_HTTP_CONNECTION_STRING replaces the whole string for an endpoint the pieces cannot describe. HttpBuilder() covers the examples that change one of the five keys the assembled string already sets; appending covers the rest, since no other key collides. Core_002_ConnectionStringConfiguration and Core_003_DependencyInjection keep their literals, because configuration is what they teach, and Testing_001_Testcontainers starts its own server. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ClickHouse.Driver references ClickHouse.Driver.Tcp with PrivateAssets="all", so the native client's types do not reach a consumer of the driver project; the examples reference it directly. CHTCP0001 is suppressed project-wide, and Tcp/README.md explains the opt-in a consumer has to make for themselves. An example's transport comes from its class name. Every example shares one namespace, so a native-protocol example cannot reuse an HTTP example's class name, and the Tcp prefix that keeps them apart also says which endpoint it needs. --http and --tcp select on it, and --list takes it too. ExamplePreflight reaches those endpoints once before anything runs and reports the endpoint, the reason, and the variables that change it. It exits non-zero rather than skipping, because CI runs the suite with no filter and a skip would leave the run green having exercised nothing. Asking for a transport checks that transport, so --tcp reports whether port 9000 answers even with no example written yet. The workflow's server publishes 9000 alongside 8123, and its paths filter covers the Tcp project. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four under examples/Tcp/Core/: constructing a client and running the four operations; the connection string's native key set and deriving an options variant; AddClickHouseTcpDataSource with keyed registrations and who disposes the pool; and the same task written over both transports, with the call-for-call mapping and an honest account of what the native client cannot do. An example that talks to both interfaces needs both endpoints checked, which its class-name prefix does not say, so ExampleInfo carries RequiredTransports alongside the transport it is filed under. Tcp/README.md said every public type of the native client is experimental. Six are; the options record, the builder, Block, the columns and the exceptions are not. It also now records that per-query roles and databases have no native equivalent, and that a timestamp column reaches the row and block tiers as the integer the wire carried. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
IDateTimeColumn and ITimeColumn reach the timezone and scale, so the row tier is the only one that hands back the raw count. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four under examples/Tcp/Read/: the three read tiers with what each allocates, measured over 200,000 rows; the block tier in depth, including the borrowed-span contract and the temporal column interfaces; parameter binding with the three traps that cost real time; and POCO reads and writes over one class. Tcp_001 presented QueryAsync<T> as the only way to get a calendar value out of a timestamp column, which the block tier's IDateTimeColumn now also does. Tcp/README.md's list of experimental types omitted the AddClickHouseTcpDataSource overloads and counted the session twice. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two under examples/Tcp/Write/: the columnar insert tier, matching by name with a named subset and the server defaulting the rest; and composite writes, covering both accepted Array(T) shapes and re-inserting a column read straight out of a block for five composite families with nothing rebuilt. These close the write-path documentation debt the plan recorded: the two array shapes, the non-nullable-row rule, and the dense round trip now have a runnable demonstration each. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five under examples/Tcp/Types/: the scalar CLR map including the 256-bit integers and when a Decimal needs ClickHouseTcpDecimal; the temporal types, with the presentation timezone measured under five session_timezone values and the insert side of the same model; composite reads through the typed views and the geo aliases as ValueTuple; Variant, Dynamic and JSON, including the normalization that makes text in differ from text out; and QBit's bit planes, rebuilding a vector at the precision L2DistanceTransposed would use. The QBit and strided-QBit cases this server refuses print its own refusal and the version they need rather than being left out. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four under examples/Tcp/Connection/: sessions, showing what one pinned connection buys and that disposal closes rather than pools it; the pool's seven knobs with the cap, the PoolTimeout expiry, the sweep and Lifo against Fifo all measured; TLS configuration and the mistakes the constructor refuses; and the timeouts, separating ReadTimeout's idle deadline from a total time limit. Pool state is not reportable, so Tcp_017 reads the pool's own log lines and polls system.processes from a second client for the concurrency figure: from the client side a query waiting for a slot and a query running look alike. Tcp_018 needs a TLS endpoint for its last section and CI has none, so that part is opt-in on CLICKHOUSE_TCP_TLS_CONNECTION_STRING and says why it skipped. The rest needs no server: the port derivation, and the refusals. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Six under examples/Tcp/Advanced/: settings precedence and query ids; the progress, profile-info and profile-event callbacks, with the timeline showing progress arriving while the rows do; cancellation, including what the server records and why the connection is closed rather than pooled; the exception hierarchy and which errors are worth retrying; compression measured in bytes on the wire; and the server info a caller gates behaviour on. Tcp_024 measures wire size through a counting socket and prints no wall-clock ranking: over loopback there is no bandwidth to save, so timing there measures the CPU cost and none of the benefit. The bytes say the client's codec does not decide what the server sends - a zstd client reads an LZ4 response, because the query packet carries one flag and the server frames with its own network_compression_method - while it does decide what an insert writes. Tcp_020 and Tcp_023 each hold a connection busy to demonstrate a real transient failure. Both start the holder with AsTask() rather than Task.Run, so the query packet is sent on the caller's thread: queued behind a busy thread pool, the holder could start second and be the query that was refused. Tcp_023 also lets only the retrying side declare the concurrency limit, so which query loses is fixed by construction rather than by ordering. Tcp_020 runs its unparseable-setting-value case on a throwaway client. That error is raised while the server parses the settings list, so it closes the socket and the pool keeps the connection, and the next operation on it fails. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five under examples/Tcp/Observability/: the three log categories and what each reports, with two filter sets; OpenTelemetry, collecting the client's spans and showing the server's own joining the same trace; the three block-shaped callbacks Tcp_021 does not cover, copying out of a borrowed block; PingAsync as a health check, measured against SELECT 1; and a throwaway container reached over the native port. Tcp_030 waits on a handshake rather than on either wait strategy. Neither probe tests the native protocol - the HTTP one tests the other listener, and UntilInternalTcpPortIsAvailable only says the port is bound - so on a loaded machine the first handshake is refused after both have passed. Forced with a no-condition strategy that took 43 attempts over 4.4 seconds. Only a transport exception is retried: a server exception means it answered and rejected us. Tcp/README.md now records W3C trace context propagation under what only the native client does. The client sends the current Activity's ids with each query, so the server's spans join the caller's trace; the HTTP transport sends no traceparent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Runner and configuration:
- A whole-string CLICKHOUSE_{HTTP,TCP}_CONNECTION_STRING override now reaches
ExampleConfig.HttpBuilder()/TcpBuilder(), which rebuilt from the component
variables and so pointed an example at a different server than preflight
checked. Host/HttpPort/TcpPort are private; HttpEndpoint/TcpEndpoint answer
for the effective endpoint.
- --http and --tcp no longer include the three examples that need a cluster,
Cloud credentials or a token. --list still shows them, marked.
- A transport flag now narrows --filter instead of being ignored.
- The two Testcontainers examples start their own server, so preflight no
longer holds them up on the configured endpoint.
- TcpOpenTelemetry queries both interfaces, so it is registered as
cross-transport and both endpoints are checked.
Native examples:
- Gate QBit (Tcp_015) and Geometry (Tcp_013) on 25.11, and QBit on the HTTP
side too: both fail outright on 25.8, the floor of the support matrix.
- Every fixed-name table is dropped before it is created, so a run interrupted
before its finally does not break the next one.
- Drop the trailing break from 42 single-block loops. Stopping a StreamAsync
enumeration early makes the client cancel the query and discard the
connection, which is not the pattern to teach.
- Tcp_012 pins session_timezone=UTC in the comparison that reads a
Kind=Unspecified DateTime, which otherwise means something different per
server.
- Tcp_017's concurrency marker and the deliberately-missing table names in
Tcp_026 and Tcp_027 are unique per run.
- Tcp_020 polls its own table instead of issuing SYSTEM FLUSH ASYNC INSERT
QUEUE, which flushes every client's pending inserts.
- WaitUntilRunning throws instead of returning after its last poll, and
Tcp_023 reports retry exhaustion instead of throwing out of the example.
- Tcp_016 creates its role and user inside the try that drops them.
- Tcp_030 retries only a socket failure; the same exception type also carries
TLS and DNS failures.
Corrected claims: values in an object[] row are boxed only when the column is
a value type; MaxRowsPerBlock is block geometry and MaxSendBufferBytes is the
memory bound; toString of a NULL is NULL, not an empty string; BFloat16, IPv4,
IPv6 and String are CLR surfaces rather than the wire bytes; a parameterized
query below revision 54459 is refused by the client before it is sent; a
TimeoutException does not say whether the pool, the dial or the read timed out;
a callback that copies values can still throw; GetServerInfoAsync describes the
connection it borrowed.
Also fixes three HTTP examples that built settings from Host = "localhost"
instead of ExampleConfig.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five native examples and the two indexes now teach what the block tier can do: - Tcp_001_ScalarTypes writes an Enum from a label and a String from raw bytes, then reads both back through IEnumColumn, IStringColumn, and Block.ReadAs. - Tcp_002_DateTimeAndTimezones reads a whole DateTime column as DateTimeOffset with Block.ReadAs. - Tcp_003_CompositeRead walks every column through the non-generic composite views, with no element type known. - Tcp_004_VariantDynamicJson names each row's alternative from IVariantColumn.TypeNames instead of printing bare discriminators. - Tcp_002_CompositeWrites builds the flat array shape with ClickHouseTcpColumn.CreateArray and asks ClickHouseTcpTypes.CanWrite which CLR types a target column accepts. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each of these hand-rolled something the client can now be asked for, or asserted something it can now show. - ITupleColumn.FieldNames is empty rather than null for an unnamed tuple, so the null-coalesce is dead. - ClickHouseTcpClientOptions.ResolvedPort replaces two hand-written derivations of the default port, one of which hard-coded 9000 and 9440. - ClickHouseTcpInsertOptions.DeduplicationToken replaces the raw setting. - ClickHouseTcpServerInfo separates the three protocol revisions, so the server info example prints which one is in force rather than one unlabelled number. - A query with no QueryId gets one from the client, which the log lines carry. - OnBlockWritten shows the block sizing MaxRowsPerBlock produced, and measures that the client's codec governs what an insert writes: 160,023 bytes go out as 80,083 under LZ4 and 33,850 under ZSTD. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
alex-clickhouse
force-pushed
the
tcp/epic-t3-examples
branch
from
September 4, 2026 09:36
982eb6c to
da30261
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Epic T3. Adds 30 runnable examples for the native (TCP) client and reorganises
examples/so the twotransports live side by side.
Stacked on #594 (
tcp/epic-u1-block-projection), which acts on section B of the follow-up list these examples produced. Below that, #592 froze the surface they use.Replaces #593, closed so the stack reads in number order.
What is here
30 native examples in seven directories, numbered from
001within each one, as the HTTP examples are:Tcp/CoreTcp/ReadTcp/WriteTcp/TypesVariant/Dynamic/JSON,QBitTcp/ConnectionTcp/AdvancedTcp/ObservabilityEach is a
public static classwith apublic static Task Run(), registered inProgram.csand listed inthe README, and each prints what it did and what came back rather than asserting.
The 54 HTTP examples move to
examples/Http/(pure renames) and now take their server from one place.Reorganisation
examples/ExampleConfig.csis the single source of the endpoint.CLICKHOUSE_HOST,CLICKHOUSE_HTTP_PORT,CLICKHOUSE_TCP_PORT,CLICKHOUSE_USER,CLICKHOUSE_PASSWORD,CLICKHOUSE_DATABASE, orCLICKHOUSE_{HTTP,TCP}_CONNECTION_STRINGto replace a whole string. Fourexamples are exempt and say so: two teach configuration, two start their own server.
examples/ExamplePreflight.csreaches the endpoints the selected examples need before any of themruns, and prints what to fix. It exits non-zero rather than skipping, because CI runs the suite unfiltered
and a skip would leave the run green with nothing exercised.
dotnet run -- --tcpand--httprun one transport;--listtakes them too.examples/AGENTS.md(plus aCLAUDE.mdpointing at it) writes down the five steps for adding anexample — the registration in
Program.csis the one that gets forgotten — the naming rule that keeps thetwo transports' class names apart, and the table-naming and endpoint rules.
examples/Tcp/README.mdcovers the port, theCHTCP0001opt-in, what the native client cannot do(no format other than Native, no ADO.NET and so no ORM, no JWT, no custom headers, no per-query role or
database) and what only it does (blocks, real sessions, live progress, block compression,
QBitplanes,W3C trace context).
.github/workflows/examples.ymlpublishes 9000 and watchesClickHouse.Driver.Tcp/**.Review
The branch was reviewed per directory. Everything the review raised about
examples/is fixed in3f88b6e, the last commit — four P1s in the runner (a whole-string override that did not reach thebuilders;
--httprunning the three examples that need a cluster, Cloud or a token; a transport flagignored alongside
--filter; preflight holding up the two self-contained examples), version gates forQBitandGeometry, which fail outright on 25.8, and nine prose claims that were wrong.Two behaviours worth calling out, because they are the kind of thing an example teaches by accident:
breakafter their first block. Stopping aStreamAsyncenumeration early makesthe client cancel the query and discard the connection, so every one of those loops was demonstrating the
abandon path. All 42 read a single-block result, so they now drain.
finallyused to break the next run atCREATE TABLE, and the runner pauses between examples wheninteractive, so that is a common way to leave one behind.
What the review found about the driver rather than the examples is recorded as follow-ups, not fixed here:
TimeoutExceptioncannot be attributed to the pool, the dial or the read, so a health check cannot honestlyseparate "busy" from "down"; and
GetServerInfoAsyncdescribes the connection it borrowed, which a versiongate can outlive.
The surface #594 adds
Five examples use it, so the block tier is taught with the projection and composition it has:
Types/Tcp_001_ScalarTypeswrites anEnumfrom a label and aStringfrom raw bytes, then reads bothback through
IEnumColumn,IStringColumn, andBlock.ReadAs. Its earlier advice was to change theschema to
FixedString(N)for bytes.Types/Tcp_002_DateTimeAndTimezonesreads a wholeDateTimecolumn asDateTimeOffset.Types/Tcp_003_CompositeReadwalks every column through the non-generic composite views, naming the trapthat a wrong type argument on a typed view compiles and never matches.
Types/Tcp_004_VariantDynamicJsonnames each row's alternative fromIVariantColumn.TypeNamesrather thanprinting bare discriminators.
Write/Tcp_002_CompositeWritesbuilds the flat array shape withClickHouseTcpColumn.CreateArray, and asksClickHouseTcpTypes.CanWritewhich CLR types a target column accepts instead of listing them by hand.Verification
Ran against a live 26.6.1.1193:
dotnet run -- --tcp, 30/30, exit 0, re-run after the TCP U1: projection and composition on the block tier #594 surface went in.dotnet run, 79 examples across both transports, exit 0. Every exception in the logs is one an exampleraises on purpose.
CLICKHOUSE_TCP_PORT=1 dotnet run -- --filter tcptestcontainersstarts its container and handshakes,confirming the self-contained examples no longer depend on the configured endpoint.
No changelog fragment: this is an epic branch in the
tcp/**stack.🤖 Generated with Claude Code