Skip to content

TCP O: sessions — one pinned connection for temporary tables and SET - #578

Open
alex-clickhouse wants to merge 5 commits into
tcp/epic-k-type-aliasesfrom
tcp/epic-o-sessions
Open

TCP O: sessions — one pinned connection for temporary tables and SET#578
alex-clickhouse wants to merge 5 commits into
tcp/epic-k-type-aliasesfrom
tcp/epic-o-sessions

Conversation

@alex-clickhouse

Copy link
Copy Markdown
Collaborator

Stacked on #577. Review only the last two commits; the base carries the rest of the stack.

Epic O of the native-TCP client. A session pins one pooled connection for its lifetime, so the state a single connection owns — temporary tables, and the settings a SET changes — is still there for the next operation.

await using IClickHouseTcpSession session = await client.OpenSessionAsync(ct);

await session.ExecuteAsync("CREATE TEMPORARY TABLE tmp (x UInt32)");
await session.InsertAsync("INSERT INTO tmp (x) VALUES", columns);
await foreach (Row row in session.QueryAsync<Row>("SELECT * FROM tmp")) { }

How it works

PinnedConnectionSource is an IConnectionSource that hands out the one lease it holds. Because that is the same seam the pool plugs into, a session runs the ordinary client code over it and implements no operation of its own, so it cannot drift from the client. The inner client shares the parent's POCO registry, so a session costs no second plan compile.

The interface is split three ways. IClickHouseTcpOperations holds every operation; IClickHouseTcpClient adds OpenSessionAsync and IClickHouseTcpSession adds IsOpen. So code that only runs operations takes the base interface and accepts either, and a session cannot offer a nested OpenSessionAsync that would silently pin a different connection. The first commit is that move on its own — no behaviour change, and callers still compile because the members are inherited. The concrete session class is internal, since OpenSessionAsync returns the interface and a public sealed class nobody can construct or name in a signature would add nothing.

Disposal terminates the connection before returning the lease. That order is the guarantee, not a detail: the pool decides a returned connection's fate from its state, so one still Ready on return is one the pool keeps, carrying this session's temporary tables into an unrelated caller's queries. Disposal during an operation cannot terminate — that would return the reader's pooled buffers underneath a live read — so it aborts the transport, which is safe to race with, and the operation completes the return as it unwinds.

Liveness is tested at both ends of an operation. On release that catches what the operation did to the connection; before the next operation it catches one dropped while the session sat idle, which release cannot see because at release nothing has had time to go wrong. That is what turns the connection's internal "terminated and cannot be reused" into a session-aware message. Two traps follow, both pinned by tests: a connection mid-operation is not reusable by definition, so both callers must exclude the busy case or every busy session is condemned; and the answer must latch, or IsOpen could say the session is finished and the next operation disagree.

Known caveat, documented and pinned rather than fixed

Aborting frees an operation parked on the socket, but a stream enumerator nobody disposes is parked at its yield, so nothing resumes it, its lease is never returned, and the pool is short a slot until the client itself is disposed — while DisposeAsync returns promptly. Returning the lease from disposal instead is worse: the pool would then terminate a connection whose buffers a live operation may still point at.

The trigger is narrower than it looks. await foreach compiles to a try/finally that disposes the enumerator, so an ordinary break is fine; it takes a hand-rolled GetAsyncEnumerator that is never disposed, which already leaks a slot on the plain client. DisposeAsync_WithAStreamNobodyAdvancesOrDisposes_DoesNotGetTheSlotBack pins the behaviour at MaxPoolSize = 1.

Tests

31 new tests, all green; whole TCP suite 3123 passing.

Integration (15) asserts everything that defines a session against a real server, using a temporary table as the marker — the server scopes one to the connection that made it, so its visibility reports which connection ran a query without the client having to claim anything about its own pool. Kill-on-dispose is proved at MaxPoolSize = 1, where the next query must land on whatever the session gave back.

Unit (16) covers only what a live session cannot show: the terminate-before-return order, disposal with an operation still running, and a 200-iteration three-way race on disposal, since over-releasing the pool's permit is the failure that would not announce itself.

Each load-bearing behaviour was mutated away and confirmed to fail exactly its own tests. One mutation caught a weak assertion: the concurrency test was satisfied by the connection's busy guard rather than the session's, so it now pins the session's own message.

Notes for review

  • No changelog fragment, per the practice for this stack (R3 sweeps it at the end).
  • No docs change: nothing under docs/ covers the TCP client yet.
  • A follow-up is filed to add a session case to the Cloud suite. Nothing exercises a session over TLS, which is where the one accepted risk here would surface: the liveness test polls the raw socket, and under TLS a record carrying no application data can make a healthy connection look readable. The pool pays a reconnect for that; a session pays the state it exists for. Same probability, never observed against ClickHouse.

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds native TCP sessions that pin one pooled connection to preserve temporary tables and SET state.

Changes:

  • Splits shared operations into IClickHouseTcpOperations.
  • Adds session lifecycle and pinned-connection handling.
  • Adds comprehensive integration and concurrency tests.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
PinnedConnectionSource.cs Manages pinned leases and disposal races.
IClickHouseTcpSession.cs Defines the public session contract.
IClickHouseTcpOperations.cs Extracts shared TCP operations.
IClickHouseTcpClient.cs Adds session creation.
ClickHouseTcpSession.cs Delegates session operations to the client.
ClickHouseTcpClient.cs Creates sessions and shares POCO plans.
ClickHouseTcpSessionIntegrationTests.cs Tests session behavior against ClickHouse.
PinnedConnectionSourceTests.cs Tests lease lifecycle and concurrency.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread ClickHouse.Driver.Tcp/Client/IClickHouseTcpSession.cs Outdated
Comment thread ClickHouse.Driver.Tcp/Client/IClickHouseTcpSession.cs Outdated
@alex-clickhouse
alex-clickhouse marked this pull request as ready for review August 19, 2026 14:48
@codecov

codecov Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-o-sessions branch 2 times, most recently from 4c07e99 to 53e1052 Compare August 22, 2026 17:06
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-o-sessions branch 2 times, most recently from b21612a to 3044804 Compare August 26, 2026 15:40

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d64b565. Configure here.

@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-o-sessions branch 2 times, most recently from d691d01 to 82674ce Compare August 28, 2026 12:18

@kavirajk kavirajk left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-o-sessions branch 2 times, most recently from a45877e to fa44fe8 Compare August 28, 2026 16:32
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-o-sessions branch 2 times, most recently from 3e2bb3e to d590438 Compare August 31, 2026 17:20
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-o-sessions branch 2 times, most recently from d70dc37 to 82ff960 Compare September 2, 2026 11:07
@alex-clickhouse
alex-clickhouse force-pushed the tcp/epic-o-sessions branch 2 times, most recently from 6458935 to 7ab080b Compare September 3, 2026 14:11
alex-clickhouse and others added 5 commits September 4, 2026 11:17
Every member moves to a new IClickHouseTcpOperations, which
IClickHouseTcpClient now extends and adds nothing to yet. No behaviour
changes and no caller changes: the members are inherited, so code
written against IClickHouseTcpClient still compiles.

This is preparation for sessions. A session runs the same operations as
a client but over one pinned connection, so both need the operation
surface while only one of them can hand out sessions. Naming that
surface separately lets code which merely runs operations take either.

Co-Authored-By: Claude <noreply@anthropic.com>
A session pins one pooled connection for its lifetime, so the state a
single connection owns — temporary tables, and the settings a SET
changes — is still there for the next operation. Open one with
client.OpenSessionAsync and dispose it to end it.

It is built on PinnedConnectionSource, an IConnectionSource that hands
out the one lease it holds. Because that is the same seam the pool
plugs into, a session runs the ordinary client code over it and
implements no operation of its own, so it cannot drift from the client.
The inner client shares the parent's POCO registry, so a session costs
no second plan compile.

Disposal terminates the connection *before* returning the lease. That
order is the guarantee: the pool decides a returned connection's fate
from its state, so one still Ready on return is one the pool keeps,
carrying this session's temporary tables into an unrelated caller's
queries. Disposal during an operation cannot terminate — that would
return the reader's pooled buffers underneath a live read — so it
aborts the transport, which is safe to race with, and the operation
completes the return as it unwinds.

The pinned connection is tested for liveness at both ends of an
operation. On release that catches what the operation did to it; before
the next operation it catches a connection dropped while the session
sat idle, which is the case release cannot see because at release
nothing has had time to go wrong. Two traps that follow, both pinned by
tests: a connection mid-operation is not reusable by definition, so
both callers must exclude the busy case or every busy session is
condemned, and the answer must latch or IsOpen could say the session is
finished and the next operation disagree.

Known caveat, documented and pinned rather than fixed: aborting frees
an operation parked on the socket, but a stream enumerator nobody
disposes is parked at its yield, so nothing resumes it and its slot
stays out until the client is disposed. The plain client has the same
hazard for a leaked enumerator.

Co-Authored-By: Claude <noreply@anthropic.com>
The IsOpen contract said the session only tests its connection at the end of
an operation. It tests at both ends, and a test pins that, so the doc was
wrong. Also add the abandoned-enumerator caveat to the public session
contract, where a caller can see it, instead of only on the internal source.

Trim the comments on the new session code by about a third. The facts stay:
the terminate-before-return order, why disposal aborts instead of closing,
the pool slot an abandoned enumerator holds, and the TLS false-positive risk.
The reasoning that belongs in the pull request description goes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants