TCP T3: usage examples for the native client

.github/workflows/examples.yml

@@ -7,13 +7,15 @@ on:
       - "examples/**"
       - "ClickHouse.Driver/**"
       - "ClickHouse.Driver.Common/**"
+      - "ClickHouse.Driver.Tcp/**"
       - ".github/workflows/examples.yml"
   pull_request:
     branches: [main]
     paths:
       - "examples/**"
       - "ClickHouse.Driver/**"
       - "ClickHouse.Driver.Common/**"
+      - "ClickHouse.Driver.Tcp/**"
       - ".github/workflows/examples.yml"
   workflow_dispatch:
     inputs:
@@ -36,6 +38,8 @@ jobs:
         image: clickhouse/clickhouse-server:latest
         ports:
           - 8123:8123
+          # The native protocol listens separately; the Tcp examples need it.
+          - 9000:9000
         env:
           CLICKHOUSE_DB: test
           CLICKHOUSE_SKIP_USER_SETUP: "1"

ClickHouse.Driver.Tests/ADO/ClickHouseRawResultDecompressionTests.cs

@@ -195,7 +195,7 @@ public async Task ReadDecompressedStreamAsync_WithUnsupportedCodec_LeavesTheBody
 
     /// <summary>
     /// Contrast case: the four original members are verbatim pass-throughs and must stay that way —
-    /// <c>examples/Select/Select_005_CompressedRawExport.cs</c> writes the compressed bytes to a file.
+    /// <c>examples/Http/Select/Select_005_CompressedRawExport.cs</c> writes the compressed bytes to a file.
     /// </summary>
     [Test]
     public async Task TheOriginalRawResultMembers_WithCompressedResponse_ReturnTheRawBytesVerbatim()

docs/overview.mdx

@@ -1079,7 +1079,7 @@ var json = await bodyReader.ReadToEndAsync();
 
 Read the returned stream to completion before it goes out of scope, as above. When the response *is* compressed you get a decoder created with `leaveOpen`, so disposing it leaves the response intact; when it is **not** compressed you get the HTTP content stream itself, so disposing it ends the body. Either way the `ClickHouseRawResult` owns the response — don't call its other read members after the stream has been disposed. Disposing the `ClickHouseRawResult` is always required and on its own sufficient: it releases both the response and any decoder inserted here (decoders hold pooled buffers). The `await using` above is therefore optional, and safe to keep. Repeated sequential calls hand back the same stream; the type is not safe to use concurrently.
 
-See [Select_007_ResponseCompression.cs](https://github.com/ClickHouse/clickhouse-cs/blob/main/examples/Select/Select_007_ResponseCompression.cs) for a runnable example.
+See [Select_007_ResponseCompression.cs](https://github.com/ClickHouse/clickhouse-cs/blob/main/examples/Http/Select/Select_007_ResponseCompression.cs) for a runnable example.
 
 #### Insert (request) compression {#insert-compression}
 
@@ -2741,7 +2741,7 @@ SqlMapper.AddTypeHandler(new BigIntegerHandler());
 SqlMapper.AddTypeHandler(new IpAddressHandler());
 ```
 
-See the [Dapper example](https://github.com/ClickHouse/clickhouse-cs/blob/main/examples/ORM/ORM_001_Dapper.cs) for an example type handler implementation.
+See the [Dapper example](https://github.com/ClickHouse/clickhouse-cs/blob/main/examples/Http/ORM/ORM_001_Dapper.cs) for an example type handler implementation.
 
 #### Dapper.Contrib {#dapper-contrib}
 

examples/AGENTS.md

@@ -0,0 +1,91 @@
+# Examples — contributor guide
+
+Each example is a self-contained, runnable demonstration of one topic. `dotnet run` in this directory
+runs them all against a live server, and CI does the same on every pull request that touches
+`examples/` or the driver, so an example that throws fails the build.
+
+Examples are grouped by transport: `Http/` uses `ClickHouseClient` / `ClickHouseConnection` over
+HTTP, `Tcp/` uses `ClickHouseTcpClient` over the native protocol. See `Tcp/README.md` for what is
+specific to the native client.
+
+## Adding an example
+
+Five steps. Skip any one of them and the example does not run.
+
+1. **Put the file** in the transport and category it belongs to:
+   `Http/<Category>/<Category>_0NN_<Topic>.cs` or `Tcp/<Category>/Tcp_0NN_<Topic>.cs`. Take the next
+   free number in that category.
+
+2. **Declare `namespace ClickHouse.Driver.Examples;`** — not a namespace derived from the folder.
+   `ExampleRunner` matches that namespace exactly and will not find a class in any other one.
+
+3. **Make it a `public static class` with a `public static Task Run()`.** Discovery requires both.
+   The class name is what `--list` prints and what `--filter` matches, so name it after the topic
+   (`BasicUsage`), not after the file (`Core_001_BasicUsage`).
+
+   **A native-protocol example's class name must start with `Tcp`** (`TcpBasicUsage`). Every example
+   shares one namespace, so it could not reuse an HTTP example's name anyway, and that prefix is how
+   `ExampleInfo.Transport` knows which endpoint to check before running it — and what `--http` and
+   `--tcp` select on.
+
+4. **Add it to `RunAllExamples` in `Program.cs`**, under its category banner, in file-number order.
+   That list is hand-maintained so the run order and the banners stay meaningful. An example missing
+   from it still compiles, still appears in `--list`, and still runs under `--filter` — it just never
+   runs in CI, so nothing tells you when it breaks.
+
+5. **Add it to the index in `README.md`**, in the matching section, with a one-line description.
+
+Then run it (`dotnet run -- --filter <topic>`) and read the output. An example whose output does not
+teach the topic is not finished.
+
+## Never hard-code a connection string
+
+Take the server from `ExampleConfig`, which resolves environment variables over localhost defaults:
+
+- `ExampleConfig.CreateHttpClient()` / `CreateHttpConnection()` for the common case.
+- `ExampleConfig.HttpConnectionString` where a constructor takes the string itself.
+- `$"{ExampleConfig.HttpConnectionString};SomeKey=value"` to add a key the assembled string does not
+  already set. It sets `Host`, `Port`, `Username`, `Password` and `Database`, so appending any of
+  those would produce a duplicate.
+- `ExampleConfig.HttpBuilder()` to *change* one of those five. It returns a fresh builder each call.
+
+Seven examples are exempt, because configuration is their subject, they use a separate endpoint, or
+they start their own server:
+`Core_002_ConnectionStringConfiguration`, `Core_003_DependencyInjection`,
+`Auth_001_JwtAuthentication`, `Tables_003_CreateTableCloud`, `Testing_001_Testcontainers`,
+`Tcp_003_Tls`, `Tcp_005_Testcontainers`. A literal connection string inside a comment,
+shown to teach the reader what one looks like, is also fine.
+
+Ask `ExampleConfig` for the endpoint with `HttpEndpoint` or `TcpEndpoint` when an example has to name
+or dial it itself. There are no `Host`/`Port` properties: a whole-string override would not reach
+them, so an example reading one could print an endpoint it did not connect to.
+
+## Examples deliberately left out of `RunAllExamples`
+
+Three need infrastructure the CI server does not have, so they are registered nowhere and run only
+by explicit filter:
+
+- `Tables_002_CreateTableCluster` — needs a ClickHouse cluster.
+- `Tables_003_CreateTableCloud` — needs ClickHouse Cloud credentials.
+- `Auth_001_JwtAuthentication` — needs a JWT.
+
+Their class names are also in `ExampleRunner._optIn`, which is what keeps `--http` and `--tcp` from
+running them: those select by reflection, so leaving an example out of `RunAllExamples` does not on
+its own keep a transport run from picking it up.
+
+If you add an example of that kind, leave it out of `RunAllExamples`, add its class name to
+`_optIn`, and list it here. Otherwise the omission is indistinguishable from having forgotten step 4.
+
+## Style
+
+- Console output is the teaching surface. Print what you did and what came back, not just "OK".
+- Show one thing well rather than covering an API exhaustively. A reader who wants the full surface
+  reads the docs.
+- Drop any table you create, in a `finally`, and `DROP TABLE IF EXISTS` it before the `CREATE` as well. The
+  second one is what lets a run survive an earlier run that was interrupted before its `finally`.
+- Name a table `example_<topic>`, fixed, not unique per run. This project is not the test suites: it assumes
+  one suite run at a time against a server, so it does not need `CreateTableName`. Two things do need a
+  per-run `Guid`: a name the example needs the server *not* to have (a deliberately missing table), and a
+  marker an example counts in `system.processes` or `system.query_log`. A second run would otherwise make
+  the first one's measurement or expected failure wrong.
+- Comments explain why the server or the driver behaves as it does, not what the line does.

examples/CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md

examples/ClickHouse.Driver.Examples.csproj

@@ -5,13 +5,19 @@
         <TargetFramework>net10.0</TargetFramework>
         <ImplicitUsings>enable</ImplicitUsings>
         <Nullable>enable</Nullable>
+        <!-- The native-protocol client is [Experimental("CHTCP0001")]. Every Tcp/ example would otherwise
+             open with the same pragma; Tcp/README.md explains the opt-in a consumer has to make. -->
+        <NoWarn>$(NoWarn);CHTCP0001</NoWarn>
     </PropertyGroup>
 
     <ItemGroup>
         <ProjectReference Include="..\ClickHouse.Driver\ClickHouse.Driver.csproj" />
         <!-- Compression types (incl. the LZ4 codec used by the bulk-insert example) live in the
              bundled (PrivateAssets="all") Common assembly; reference it directly. -->
         <ProjectReference Include="..\ClickHouse.Driver.Common\ClickHouse.Driver.Common.csproj" />
+        <!-- Same reason: ClickHouse.Driver references Tcp with PrivateAssets="all", so the native
+             client's types do not reach a consumer of the driver project transitively. -->
+        <ProjectReference Include="..\ClickHouse.Driver.Tcp\ClickHouse.Driver.Tcp.csproj" />
     </ItemGroup>
 
     <ItemGroup>

examples/ExampleConfig.cs

@@ -0,0 +1,88 @@
+using ClickHouse.Driver.ADO;
+using ClickHouse.Driver.Tcp;
+using ClickHouse.Driver.Utility;
+
+namespace ClickHouse.Driver.Examples;
+
+/// <summary>Creates example clients from shared environment settings.</summary>
+/// <remarks>
+/// Component settings use local Docker defaults. A transport-specific connection string overrides
+/// all components. See the examples README for the supported environment variables.
+/// </remarks>
+public static class ExampleConfig
+{
+    private static readonly string Host = Env("CLICKHOUSE_HOST") ?? "localhost";
+    private static readonly ushort HttpPort = ushort.Parse(Env("CLICKHOUSE_HTTP_PORT") ?? "8123");
+    private static readonly ushort TcpPort = ushort.Parse(Env("CLICKHOUSE_TCP_PORT") ?? "9000");
+    private static readonly string Username = Env("CLICKHOUSE_USER") ?? "default";
+    private static readonly string Password = Env("CLICKHOUSE_PASSWORD") ?? string.Empty;
+    private static readonly string Database = Env("CLICKHOUSE_DATABASE") ?? "default";
+
+    /// <summary>The connection string for the HTTP transport.</summary>
+    public static string HttpConnectionString { get; } =
+        Env("CLICKHOUSE_HTTP_CONNECTION_STRING") ?? FromComponents().ConnectionString;
+
+    /// <summary>The connection string for the native protocol.</summary>
+    public static string TcpConnectionString { get; } =
+        Env("CLICKHOUSE_TCP_CONNECTION_STRING") ?? TcpFromComponents().ToString();
+
+    /// <summary>Creates an HTTP builder from the configured connection string.</summary>
+    public static ClickHouseConnectionStringBuilder HttpBuilder() => new(HttpConnectionString);
+
+    /// <summary>Creates a native builder from the configured connection string.</summary>
+    public static ClickHouseTcpConnectionStringBuilder TcpBuilder() => new(TcpConnectionString);
+
+    /// <summary>Gets the configured HTTP host and port.</summary>
+    public static (string Host, ushort Port) HttpEndpoint
+    {
+        get
+        {
+            var builder = HttpBuilder();
+            return (builder.Host, builder.Port);
+        }
+    }
+
+    /// <summary>Gets the configured native host and port.</summary>
+    public static (string Host, int Port) TcpEndpoint
+    {
+        get
+        {
+            var builder = TcpBuilder();
+
+            return (builder.Host, builder.Port ?? 9000);
+        }
+    }
+
+    /// <summary>Creates an HTTP client. The caller owns it.</summary>
+    public static ClickHouseClient CreateHttpClient() => new(HttpConnectionString);
+
+    /// <summary>Creates a native client. The caller owns it.</summary>
+    public static ClickHouseTcpClient CreateTcpClient() => new(TcpConnectionString);
+
+    /// <summary>Creates an ADO.NET connection. The caller owns it.</summary>
+    public static ClickHouseConnection CreateHttpConnection() => new(HttpConnectionString);
+
+    private static ClickHouseConnectionStringBuilder FromComponents() => new()
+    {
+        Host = Host,
+        Port = HttpPort,
+        Username = Username,
+        Password = Password,
+        Database = Database,
+    };
+
+    private static ClickHouseTcpConnectionStringBuilder TcpFromComponents() => new()
+    {
+        Host = Host,
+        Port = TcpPort,
+        Username = Username,
+        Password = Password,
+        Database = Database,
+    };
+
+    private static string? Env(string name)
+    {
+        var value = Environment.GetEnvironmentVariable(name);
+        return string.IsNullOrWhiteSpace(value) ? null : value;
+    }
+}

examples/ExamplePreflight.cs

@@ -0,0 +1,113 @@
+using ClickHouse.Driver.Tcp;
+using ClickHouse.Driver.Utility;
+
+namespace ClickHouse.Driver.Examples;
+
+/// <summary>Checks required endpoints before any example runs.</summary>
+public static class ExamplePreflight
+{
+    private static readonly TimeSpan Timeout = TimeSpan.FromSeconds(10);
+
+    /// <summary>Checks the endpoints required by a set of examples.</summary>
+    /// <param name="examples">The examples about to run. Only their transports are checked.</param>
+    /// <returns>True when every needed endpoint answered.</returns>
+    public static Task<bool> CheckAsync(IEnumerable<ExampleRunner.ExampleInfo> examples)
+        => CheckAsync(examples.SelectMany(e => e.RequiredTransports).Distinct().ToArray());
+
+    /// <summary>Checks the named endpoints.</summary>
+    /// <param name="transports">The transports to check. Duplicates are checked once.</param>
+    /// <returns>True when every named endpoint answered.</returns>
+    public static async Task<bool> CheckAsync(params ExampleTransport[] transports)
+    {
+        var ok = true;
+
+        foreach (var transport in transports.Distinct().OrderBy(t => t))
+        {
+            string? failure = transport == ExampleTransport.Http
+                ? await CheckHttpAsync()
+                : await CheckTcpAsync();
+
+            if (failure is not null)
+            {
+                Report(transport, failure);
+                ok = false;
+            }
+        }
+
+        return ok;
+    }
+
+    private static async Task<string?> CheckHttpAsync()
+    {
+        try
+        {
+            using var cancellation = new CancellationTokenSource(Timeout);
+            using var client = ExampleConfig.CreateHttpClient();
+            await client.ExecuteScalarAsync("SELECT 1", cancellationToken: cancellation.Token);
+            return null;
+        }
+        catch (Exception ex)
+        {
+            return ex.Message;
+        }
+    }
+
+    private static async Task<string?> CheckTcpAsync()
+    {
+        try
+        {
+            using var cancellation = new CancellationTokenSource(Timeout);
+            await using var client = ExampleConfig.CreateTcpClient();
+            await client.PingAsync(cancellation.Token);
+            return null;
+        }
+        catch (Exception ex)
+        {
+            return ex.Message;
+        }
+    }
+
+    private static void Report(ExampleTransport transport, string failure)
+    {
+        // Report the effective endpoint, including whole-connection-string overrides.
+        var http = ExampleConfig.HttpEndpoint;
+        var tcp = ExampleConfig.TcpEndpoint;
+
+        var (name, endpoint, user, port, source) = transport == ExampleTransport.Http
+            ? (
+                "HTTP interface",
+                $"{http.Host}:{http.Port}",
+                ExampleConfig.HttpBuilder().Username,
+                "CLICKHOUSE_HTTP_PORT",
+                "CLICKHOUSE_HTTP_CONNECTION_STRING")
+            : (
+                "native protocol",
+                $"{tcp.Host}:{tcp.Port}",
+                ExampleConfig.TcpBuilder().Username,
+                "CLICKHOUSE_TCP_PORT",
+                "CLICKHOUSE_TCP_CONNECTION_STRING");
+
+        Console.WriteLine();
+        Console.WriteLine($"Cannot reach ClickHouse on the {name} at {endpoint} as user '{user}'.");
+        Console.WriteLine($"  {failure}");
+        Console.WriteLine();
+
+        if (transport == ExampleTransport.Tcp)
+        {
+            Console.WriteLine(
+                $"  The native protocol listens on port 9000 by default, " +
+                $"not on the HTTP port ({http.Port}).");
+            Console.WriteLine();
+        }
+
+        Console.WriteLine("  Start a server with both ports published:");
+        Console.WriteLine(
+            "    docker run -d --name clickhouse-server " +
+            "-p 8123:8123 -p 9000:9000 clickhouse/clickhouse-server");
+        Console.WriteLine();
+        Console.WriteLine("  Or point the examples somewhere else:");
+        Console.WriteLine($"    CLICKHOUSE_HOST, {port}, CLICKHOUSE_USER, CLICKHOUSE_PASSWORD, CLICKHOUSE_DATABASE");
+        Console.WriteLine($"    {source} replaces the whole connection string.");
+        Console.WriteLine();
+    }
+}