Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions context/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,13 +103,13 @@ Tokio and futures-io expose different AsyncRead and AsyncWrite traits. `tokio-ut

Entering a Tokio runtime context provides access to its services; entering alone does not drive the runtime. Some mixed execution is possible when the required services are running, but must be established for the concrete APIs being used. Do not advertise universal Tokio compatibility from a Waker or stream adapter alone.

The optional `scheduler::tokio` adapter implements Socket, File, Clock and Spawn against an existing runtime. It preserves scheduler task ownership and joins owned task destruction on asynchronous shutdown. The same generic TCP program runs on Socketry and Tokio. The adapter scopes runtime context to individual polls when registering resources; its futures can also be polled by Socketry workers while Tokio drives the underlying services. Socketry contextual lookups still identify Socketry execution; portable code passes handles explicitly.
The optional `scheduler::tokio` adapter implements SocketScheduler, FileScheduler, Clock and Spawn against an existing runtime. It preserves scheduler task ownership and joins owned task destruction on asynchronous shutdown. The same generic TCP program runs on Socketry and Tokio. The adapter scopes runtime context to individual polls when registering resources; its futures can also be polled by Socketry workers while Tokio drives the underlying services. Socketry contextual lookups still identify Socketry execution; portable code passes handles explicitly.

For existing libraries tied to Tokio, either keep their work on Tokio and bridge owned messages/results, or provide the particular trait adapter they consume. Avoid a broad imitation of Tokio's API.

## Address resolution

`AddressResolution` is independent of `Socket` and returns `Vec<AddressInfo>` using an optional hostname, numeric or named service, typed `AddressHints`, and explicit cancellation. Hints select family, socket type/protocol, passive binding, numeric-only hosts, and configured families during native resolution. Each candidate retains socket creation parameters, including raw or platform-specific values when transports are unspecified. Typed resolver errors retain native codes separately from errno. Native and Tokio schedulers use their own bounded blocking pools, sharing capacity with `spawn_blocking`; the simulator uses exact-name fixtures. System lookups preserve native `getaddrinfo`/`GetAddrInfoW` behavior, including hosts-file policy. Cancellation can prevent queued execution, but started lookups preserve their actual result and retain their host, service, and capacity until completion, even after waiter abandonment. Await them before shutdown for graceful completion. Connection retries, DNS-specific APIs, caching, canonical names, and further resolver flags remain above or outside this initial capability.
`AddressResolution` is independent of `SocketScheduler` and returns `Vec<AddressInfo>` using an optional hostname, numeric or named service, typed `AddressHints`, and explicit cancellation. Hints select family, socket type/protocol, passive binding, numeric-only hosts, and configured families during native resolution. Each candidate retains socket creation parameters, including raw or platform-specific values when transports are unspecified. Typed resolver errors retain native codes separately from errno. Native and Tokio schedulers use their own bounded blocking pools, sharing capacity with `spawn_blocking`; the simulator uses exact-name fixtures. System lookups preserve native `getaddrinfo`/`GetAddrInfoW` behavior, including hosts-file policy. Cancellation can prevent queued execution, but started lookups preserve their actual result and retain their host, service, and capacity until completion, even after waiter abandonment. Await them before shutdown for graceful completion. Connection retries, DNS-specific APIs, caching, canonical names, and further resolver flags remain above or outside this initial capability.

## I/O and io\_uring

Expand Down Expand Up @@ -137,7 +137,9 @@ Tokio-uring is useful reference code for owned-buffer operations. It has its own

The implementations live under `scheduler/selector/`, selected with Cargo features and target configuration. epoll, kqueue and iocp expose shared async-io readiness operations; the process-wide reactor owns persistent registrations. Windows uses IOCP/AFD socket readiness. Regular-file fallback operations use a blocking pool, because readiness does not make ordinary file I/O nonblocking.

`File::file_stat` queries an open descriptor and returns an owned associated metadata type. `FileMetadata` defines portable optional accessors for type, size, and access/modification/creation times, allowing native-backed results and simple simulated records. Readiness/Tokio wrap `std::fs::Metadata`; io_uring retains Linux `statx` directly in the request allocation, with native fields exposed on the concrete type. Avoid eager field normalization and additional output allocations. Returning by value keeps the interface simple; copy elimination is an optimization rather than a contract. The file and output survive waiter destruction until backend access ends. Metadata does not change the cursor or order concurrent modifications, and path-based lookup remains separate.
`FileStat::file_stat` queries an open descriptor and returns an owned associated metadata type. `FileMetadata` defines portable optional accessors for type, size, and access/modification/creation times, allowing native-backed results and simple simulated records. Readiness/Tokio wrap `std::fs::Metadata`; io_uring retains Linux `statx` directly in the request allocation, with native fields exposed on the concrete type. Avoid eager field normalization and additional output allocations. Returning by value keeps the interface simple; copy elimination is an optimization rather than a contract. The file and output survive waiter destruction until backend access ends. Metadata does not change the cursor or order concurrent modifications, and path-based lookup remains separate.

File operation traits share `FileOperation::File`; `FileStat` declares `Metadata`. `FileScheduler` is an automatically implemented aggregate of the core file capabilities, while `FileAllocate` and `FileAdvice` remain optional and depend only on `FileOperation`. Use narrow bounds for generic consumers and import individual operation traits for concrete method calls. The corresponding `SocketOperation` split is agreed follow-up work; `SocketScheduler` still declares its operations directly.

The Linux `io-uring` feature selects a dedicated ring for socket/file reads and writes, scatter/gather message sends/receives, and `file_sync`, plus descriptor-based `file_stat` using required `Statx` support. `file_truncate` uses the ring when its opcode is available, with a blocking-file fallback on older kernels. Optional `FileAllocate` adds Linux range reservation with `AllocationMode::Extend` or `KeepSize`, using a probed `Fallocate` request or the blocking syscall fallback. Other platforms currently omit that capability. Synchronization, allocation, and native truncation retain the file without a payload buffer and use the same original-completion retirement contract as transfers. Connect, accept, explicit readiness waits and sleep still use async-io. Initialization probes required opcodes and reliable completion overflow; it returns an error if unavailable rather than silently changing implementation. Shutdown cancels and drains original completions. Operation identifiers are never reused, and cancellation completions cannot release original buffers. Unexpected selector failure retains resources whose kernel lifetime cannot be established and panics their waiting operations. Normal cancellation does not retain completed resources.

Expand Down Expand Up @@ -182,7 +184,7 @@ The algorithm's existing Ruby performance motivates the port. Rust performance c
1. Record boundaries and reusable Rust conventions (implemented).
2. Replace stackful execution with async-task and Crossbeam worker queues (implemented). Keep the coroutine prototype in its saved branch.
3. Implement primitive task dispatch, scheduler ownership, cancellation and shutdown (implemented). Runtime-independent cooperative signals and deferred cancellation are implemented; cancellation-aware file/socket operations and transferable task ownership are implemented. Clock cancellation is implemented; exclusive file/socket close, directional socket shutdown, and bounded submission-only cleanup are implemented; optional Linux file advice shares the bounded cleanup admission and draining contract. Higher-level task groups are outside the executor for now.
4. Establish minimal clock and I/O contracts with concrete consumers (implemented with Socket, File, Clock and a portable TCP example).
4. Establish minimal clock and I/O contracts with concrete consumers (implemented with SocketScheduler, FileScheduler, Clock and a portable TCP example).
5. Implement native readiness and the Tokio adapter, running the same consumers with both (implemented). Native sleep uses async-io until scheduler integration with `socketry-timers`.
6. Implement io\_uring's owned-buffer lifecycle, socket/file operations, cancellation and runtime probing (implemented). Improve operation reuse, buffer registration and submission backpressure in subsequent work.
7. Port the timer queue with upstream attribution and deterministic verification (implemented in `socketry-timers`). Integrate a driven sleep future with the scheduler in subsequent work; see the [timer package](../crates/timers/readme.md) and [benchmarks](../crates/timers/context/benchmarks.md).
Expand Down
2 changes: 1 addition & 1 deletion context/gaps.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The executor does not expose a Barrier or require a group/owner argument. Librar

## 3. The I/O boundary is still narrow

`Socket` uses one associated resource type for unconnected, bound, listening, connected, and datagram sockets. It exposes `socket_open`, `socket_bind`, syscall-like `socket_listen`, `socket_connect`, `socket_accept`, address inspection, reads, writes, and readiness. The initial API supports IPv4/IPv6 and stream/datagram sockets with optional TCP/UDP protocol selection. OS-resource import is a separate `SocketRegistration` capability. Socket close/shutdown and their admission-only variants are implemented; typed option get/set supports address reuse, keepalive, and TCP no-delay; `SocketMessages` implements owned scatter/gather payloads, addresses, peeking, routing flags, and receive truncation; ancillary data and further options remain future work. `File` opens associated resources and borrows them for seeking, truncation, synchronization, shared-cursor and positioned I/O. Explicit close/submission now consumes an exclusively owned handle, and optional `FileAllocate` supplies Linux allocation and `FileAdvice` supplies hints with completion/submission forms. `file_stat` returns owned associated metadata with native storage and portable accessors; path-based metadata remains future work. File/socket operations support cancellation and owned buffer views. `Clock` supplies monotonic time and cancellable absolute and relative waits.
`SocketScheduler` uses one associated resource type for unconnected, bound, listening, connected, and datagram sockets. It exposes `socket_open`, `socket_bind`, syscall-like `socket_listen`, `socket_connect`, `socket_accept`, address inspection, reads, writes, and readiness. The initial API supports IPv4/IPv6 and stream/datagram sockets with optional TCP/UDP protocol selection. OS-resource import is a separate `SocketRegistration` capability. Socket close/shutdown and their admission-only variants are implemented; typed option get/set supports address reuse, keepalive, and TCP no-delay; `SocketMessages` implements owned scatter/gather payloads, addresses, peeking, routing flags, and receive truncation; ancillary data and further options remain future work. `FileScheduler` opens associated resources and borrows them for seeking, truncation, synchronization, shared-cursor and positioned I/O. Explicit close/submission now consumes an exclusively owned handle, and optional `FileAllocate` supplies Linux allocation and `FileAdvice` supplies hints with completion/submission forms. `file_stat` returns owned associated metadata with native storage and portable accessors; path-based metadata remains future work. File/socket operations support cancellation and owned buffer views. `Clock` supplies monotonic time and cancellable absolute and relative waits.

`Blocking` now adds bounded asynchronous admission for owned synchronous closures, implemented through dedicated native and Tokio pools. `AddressResolution` uses the same bounded pools for system hostname lookup and returns owned address/type/protocol candidates with numeric or named services and native resolver hints; the simulator uses an explicit table. Started system lookups finish independently of cancellation. Existing file fallbacks still use backend pools. Ancillary data, further socket options, randomness, and processes remain outside the portable boundary. Deadline policies can now compose `Clock` and cancellation; higher-level timeout helpers remain separate. Ordinary synchronization uses futures and wakers above scheduler primitives. Expand capabilities as real consumers need them, keeping the application's implementation choice explicit. A broader boundary would also allow more operating-system behavior to be substituted in deterministic tests.

Expand Down
Loading
Loading