Skip to content

Reduce ulitebox broker startup latency - #1494

Closed
Weidong Cui (wdcui) wants to merge 4 commits into
uliteboxfrom
wdcui/ulitebox/startup-time
Closed

Weidong Cui (wdcui) wants to merge 4 commits into
uliteboxfrom
wdcui/ulitebox/startup-time

Conversation

@wdcui

@wdcui Weidong Cui (wdcui) commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

This PR reduces the startup latency of Linux-on-Linux ulitebox runs with the out-of-process broker (median wall time, base → this PR: static hello 9.1 → 6.6 ms, dynamic hello 21.6 → 13.5 ms, bash -c true 27.4 → 17.0 ms, python3 -c pass 71.0 → 45.7 ms, node -e 0 186 → 84 ms). The broker's overlay now resolves paths with per-name lookups (a new Backend::lookup_at, overridden by the in-memory and tar backends) instead of listing and merging whole directories, while still reading each mutable upper directory as one snapshot; the tar backend memory-maps the archive to build its index and reads file contents with positional reads; broker requests are served by a lazily started blocking-handoff worker pool; the shared-memory transports copy directly to and from mapped ring and buffer memory and wake the peer only when it is waiting; runner callers take turns reading responses through a reader role instead of a dispatcher thread; shared-buffer slots are leased lowest-first; a locally closed call channel ends notification receipt cleanly instead of reporting an error; and the Linux shim gives the platform a huge-page hint (PageManagementProvider::advise_fill, MADV_HUGEPAGE on Linux userland) before eagerly copying file contents into a new mapping.

Weidong Cui (wdcui) and others added 4 commits October 3, 2026 22:49
- Resolve overlay lookups without redundant lower-layer probes.
- Map the initial-files tar archive instead of reading it into memory.
- Copy shared-memory ring and buffer data directly via peer_memory.
- Let a runner caller take the response-reader role instead of a
  dedicated responses thread; notify full-ring waiters only on the
  full-to-not-full transition.
- Run broker requests with a blocking-handoff worker pool that starts
  workers lazily, only when a request is about to block.
- Lease the lowest free shared-buffer slots so pages stay warm.
- End notification receipt cleanly when the local side closes its
  association, removing spurious exit-time errors.
- Wait for the in-process runner's control connection instead of
  sleeping between accept attempts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b5ed1835-8e32-4a35-aae1-8d8a8a276d09
- Add a defaulted `PageManagementProvider::advise_fill` hint, which the
  Linux shim gives before copying file contents into a new mapping. The
  Linux userland platform backs such ranges with transparent huge pages,
  so the copy takes one fault and clear per 2 MiB instead of per page,
  while pages past the end of the file stay lazily allocated.
- Read tar file contents through `TarStorage::read_exact_at`, which the
  userland broker's mapped archive serves with positional reads, so the
  broker no longer maps every page it serves and tears them down at exit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b5ed1835-8e32-4a35-aae1-8d8a8a276d09
Read each mutable upper directory in a single listing so a lookup's entry,
whiteout, and opaque marker come from one snapshot, list each layer once
when enumerating a directory, and end notification receipt cleanly when an
acknowledgement fails because the local call channel was closed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b5ed1835-8e32-4a35-aae1-8d8a8a276d09
The in-process runner is for development and testing only, so its startup
does not need optimizing.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b5ed1835-8e32-4a35-aae1-8d8a8a276d09
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

🤖 SemverChecks 🤖 ⚠️ Potential breaking API changes detected ⚠️

Click for details
--- failure auto_trait_impl_removed: auto trait no longer implemented ---

Description:
A public type has stopped implementing one or more auto traits. This can break downstream code that depends on the traits being implemented.
        ref: https://doc.rust-lang.org/reference/special-types-and-traits.html#auto-traits
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.51.0/src/lints/auto_trait_impl_removed.ron

Failed in:
  type TarRo is no longer UnwindSafe, in /home/runner/work/litebox/litebox/litebox_broker_core/src/fs/tar_ro.rs:50
  type TarRo is no longer RefUnwindSafe, in /home/runner/work/litebox/litebox/litebox_broker_core/src/fs/tar_ro.rs:50

--- failure method_parameter_count_changed: pub method parameter count changed ---

Description:
A publicly-visible method now takes a different number of parameters, not counting the receiver (self) parameter.
        ref: https://doc.rust-lang.org/cargo/reference/semver.html#fn-change-arity
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.51.0/src/lints/method_parameter_count_changed.ron

Failed in:
  litebox_broker_transport::pending_calls::PendingCall::wait takes 0 parameters in /home/runner/work/litebox/litebox/target/semver-checks/git-ulitebox/d5df8f676ed8d814f7090697a6c9fd5973eeb366/litebox_broker_transport/src/pending_calls.rs:111, but now takes 1 parameters in /home/runner/work/litebox/litebox/litebox_broker_transport/src/pending_calls.rs:155

--- failure method_receiver_ref_became_owned: method receiver changed from immutable reference to owned value ---

Description:
A method's receiver changed from an immutable reference to an owned value. Ownership is transferred as part of the call, which is a breaking change.
        ref: https://doc.rust-lang.org/reference/items/associated-items.html#methods
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.51.0/src/lints/method_receiver_ref_became_owned.ron

Failed in:
  litebox_broker_transport::pending_calls::PendingCall::wait now takes Self, not &Self, in /home/runner/work/litebox/litebox/litebox_broker_transport/src/pending_calls.rs:155

--- failure type_mismatched_generic_lifetimes: type now takes a different number of generic lifetimes ---

Description:
A type now takes a different number of generic lifetime parameters. Uses of this type that name the previous number of parameters will be broken.
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.51.0/src/lints/type_mismatched_generic_lifetimes.ron
Failed in:
  Struct PendingCall (0 -> 1 lifetime params) in /home/runner/work/litebox/litebox/litebox_broker_transport/src/pending_calls.rs:141

@wdcui

Copy link
Copy Markdown
Member Author

Split into #1495 (overlay lookups), #1496 (tar archive mapping, stacked on #1495), #1497 (transport fast path), #1498 (lazy broker workers), and #1499 (huge-page hint); together they are tree-identical to this branch. Note that this PR's base numbers were measured against a binary that already included the overlay and tar changes; against ulitebox (d798f1d) the combined change takes python3 -c pass from 162 to 46 ms and node -e 0 from 304 to 84 ms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant