Async refactoring: reduce code duplication

[tobixen]

I do want a library that can be used both with sync and async users, with as little code duplication as possible, and without generated code. Perhaps it was silly, perhaps "async first, autogenerated sync code" is the best solution. Anyway, Claude claimed that the "Sans-IO" design pattern would work well. Problem is that the library absolutely wasn't made Sans-IO in the first place.

We should think throughly on how to make version 4.0 more "sans-io". It's probably worth a separate ticket on "long term sans-io plans".. Right now I'm mostly concerned with "make it possible to use the library in an async way" and "refactor to reduce code duplication", and I think this is the way to go:

• All methods doing I/O should be identified. The type hints should be updated to indicate that those methods may return an async coroutine.
• Methods that first prepares something, then fires off some API call, and then processes the results should be refactored - the common code should not be duplicated. I've dealt with this some places (i.e. load_by_multiget()) by first doing the preparation, then if self.is_async_client: return self._async_load_by_multiget(xxx) and finally self._post_load_by_multiget(xxx) for post-processing.

Is this refactoring sane? Any suggestions on how to do it more properly is welcome.

[tobixen]

⚠️ This comment is AI-generated (Claude Sonnet 4.6 via Claude Code) on behalf of tobixen

Looking at the current codebase, the dominant pattern is:

def foo(self, ...):
    if self.is_async_client:
        return self._async_foo(...)
    result = <sync I/O>
    return self._post_foo(result)

async def _async_foo(self, ...):
    result = await <async I/O>
    return self._post_foo(result)

def _post_foo(self, result):
    # shared post-processing
    ...

This is essentially a manual implementation of the "prepare / execute / process" split, and it works — but it's verbose and the return type of foo() is lying: it claims to return T but may actually return Coroutine[T].

A few thoughts on alternatives:

1. Async-first + sync shim via asyncio.run() — generate sync wrappers automatically using a class decorator or metaclass. Works well if you can assume the caller isn't already inside an event loop (which isn't always true). Libraries like httpx do this (httpx.Client wraps httpx.AsyncClient). The downside: two classes, but the logic lives in one place (the async version).

2. Lazy coroutine / dual-await return object — a helper class that is both callable synchronously and awaitable. Ugly, non-obvious, hard to type. Not recommended.

3. Decorator per method — e.g. @sync_or_async that inspects self.is_async_client and dispatches. Cleaner call sites, but hides control flow in magic.

4. True Sans-IO — push all I/O out to a transport layer, methods only build requests and process responses. Cleanest long-term, but close to a full rewrite.

My take: For v4.0, approach 1 (async-first with auto-generated sync wrappers) is the most maintainable path. The current dual-mode approach in a single class accumulates complexity fast — you can see it already in collection.py with 15+ _async_* methods. Async-first means the "real" implementation lives in one place, and sync users get a thin wrapper. The main risk is breaking callers inside event loops (e.g. Jupyter), which can be mitigated by documenting that DAVClient is for pure-sync contexts.

The current pre/post refactoring (as in load_by_multiget -> _post_load_by_multiget) is a solid intermediate step regardless of which long-term direction is chosen.

⚠️ This comment is AI-generated (Claude Sonnet 4.6 via Claude Code) on behalf of tobixen

[tobixen]

⚠️ This comment is AI-generated (Claude Sonnet 4.6 via Claude Code) on behalf of tobixen

Followed up by actually reading the Sans-IO docs (https://sans-io.readthedocs.io/how-to-sans-io.html, https://snarky.ca/network-protocols-sans-i-o/, https://www.firezone.dev/blog/sans-io).

The short version: Sans-IO as described in the literature is about raw protocol implementations (HTTP/2, WebSockets, QUIC), not high-level client libraries. Its core pattern is a state machine with these APIs:

• handle_input(bytes) — feed incoming data
• poll_transmit() — drain queued outgoing data
• handle_timeout(now) / poll_timeout() — time abstraction

What this means for caldav:

The canonical Sans-IO pattern is a poor fit for a CalDAV client library. CalDAV is already built on HTTP; the library's job is high-level orchestration (discover principal → find calendar home → list calendars etc.), not protocol framing. A pure Sans-IO approach would require callers to drive each individual HTTP request manually, which is exactly the wrong interface for users.

However, the insight from Sans-IO is directly applicable: separate "what to request" from "how to execute it". The existing operations/ and protocol/ directories already capture this: QuerySpec, DAVRequest, XML builders/parsers are I/O-free. That's the useful half of Sans-IO.

The failed attempt in the git history is instructive:

Commits 43a74aa and 8a13fdd show a previous full Sans-IO attempt:

• Phase 4–5 created caldav/io/ (SyncIOProtocol, AsyncIOProtocol) and protocol/operations.py
• Phase 6 (wiring it into DAVClient/AsyncDAVClient) was never completed
• The orphaned files were eventually deleted

The commit message notes: "a separate protocol_client.py was created as an alternative, which defeats the purpose of Sans-IO (sharing code between sync/async)."

The real insight: async/await IS the natural expression of the multi-step protocol pattern

For multi-step operations (the typical CalDAV case), async/await naturally expresses the sequencing that Sans-IO state machines encode manually. The _async_foo() methods ARE effectively the Sans-IO implementation — they just aren't called that.

So the existing refactoring direction (pre-I/O prep → _async_foo → _post_foo post-processing) is the right shape. The remaining ugliness is:

1. Return types lie (foo() claims to return T but may return Coroutine[T])
2. Three methods per operation instead of one

The cleanest path to fix both: async-first, auto-generate sync via a class wrapper. The _async_foo method IS the real implementation; the sync foo calls asyncio.run(self._async_foo(...)). This is what httpx does. The whole _post_foo split disappears.

The operations/ layer remains useful for the I/O-free XML building/parsing logic regardless of which direction the async story goes.

⚠️ This comment is AI-generated (Claude Sonnet 4.6 via Claude Code) on behalf of tobixen