Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,69 @@ jobs:
--pjdfstest-dir /tmp/pjdfstest \
--profile phase5-ci

# Real macOS runtime coverage: the NFS mount path, the Seatbelt read-scoping
# posture, and the remote-tier suites all execute here, not just compile.
# macos-latest runners allow unprivileged /sbin/mount_nfs against the
# session's loopback NFS server, so no sudo is involved. A suite SKIP is
# red: a runner that cannot run these legs must fail loudly, not pass green.
macos-runtime:
name: macOS runtime gate
runs-on: macos-latest
timeout-minutes: 60
steps:
- uses: actions/checkout@v4

- name: Install Rust toolchain
uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ env.RUST_TOOLCHAIN }}

- name: Cache cargo registry
uses: actions/cache@v4
with:
path: ~/.cargo/registry
key: ${{ runner.os }}-cargo-registry-${{ hashFiles('Cargo.lock') }}

- name: Cache cargo index
uses: actions/cache@v4
with:
path: ~/.cargo/git
key: ${{ runner.os }}-cargo-index-${{ hashFiles('Cargo.lock') }}

- name: Cache cargo build
uses: actions/cache@v4
with:
path: target
key: ${{ runner.os }}-cargo-build-target-macos-runtime-${{ hashFiles('Cargo.lock') }}

- name: Put GNU coreutils on PATH
# The shell suites use sha256sum and truncate; stock macOS ships neither.
run: |
brew list coreutils >/dev/null 2>&1 || brew install coreutils
echo "$(brew --prefix coreutils)/libexec/gnubin" >> "$GITHUB_PATH"

- name: Build release binary
run: cargo +"$RUST_TOOLCHAIN" build --release -p vfs-cli

- name: NFS and Seatbelt runtime validation
run: scripts/validation/macos-nfs-git-validation.sh --vfs-bin "$PWD/target/release/vfs"

- name: Remote checkpoint e2e
env:
VFS_BIN: ${{ github.workspace }}/target/release/vfs
run: |
set -o pipefail
crates/vfs-cli/tests/test-remote-checkpoint-e2e.sh 2>&1 | tee /tmp/remote-checkpoint.log
! grep -q '^SKIP:' /tmp/remote-checkpoint.log

- name: Remote adopt e2e
env:
VFS_BIN: ${{ github.workspace }}/target/release/vfs
run: |
set -o pipefail
crates/vfs-cli/tests/test-remote-adopt-e2e.sh 2>&1 | tee /tmp/remote-adopt.log
! grep -q '^SKIP:' /tmp/remote-adopt.log

build-linux-arm64:
name: Build (Linux arm64)
runs-on: ubuntu-24.04-arm
Expand Down
32 changes: 17 additions & 15 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ Shell tests each run out of their own `mktemp -d` with trap cleanup and honor
mount, suite, or benchmark.

Full detail — Python harnesses, pjdfstest profiles, benchmark policy, the
manual macOS gate — is in [docs/TESTING.md](docs/TESTING.md).
macOS gates — is in [docs/TESTING.md](docs/TESTING.md).

## Contracts you cannot change quietly

Expand Down Expand Up @@ -226,14 +226,17 @@ Not regressions — deliberate, documented state. Don't "fix" them by accident.
The sandbox hides `$HOME` and the temp dirs, so a host path that happens to
work on the CI image can vanish on a developer's machine. Use
`sandbox_python()` for interpreters and the `GIT` env var from
`pin_distro_git()` for git, both in `scripts/validation/lib/common.py`.
Harness helpers live in that module: do not copy one into a harness, because
the copies drift and the drift is invisible until the gate runs somewhere
unusual.
* **macOS runtime validation is a manual release gate.** CI builds and clippys
on macos-latest, but `scripts/validation/macos-nfs-git-validation.sh` plus
the Seatbelt spot-checks in docs/TESTING.md must run on real hardware before
shipping a release that advertises macOS.
`pin_distro_git()` for git, both in `scripts/validation/lib/common.py`. Do
not copy helpers out of that module; copies drift, and the drift is
invisible until the gate runs somewhere unusual.
* **macOS runtime coverage is CI-real but narrower than Linux.** The
macos-latest jobs run clippy, the workspace tests, the NFS + Seatbelt
runtime validation script, and both remote-tier suites over the NFS mount
path. The session control socket and the chunk streamer exist only in the
Linux mount owner, so the live-checkpoint and streamer legs never run on a
Mac, and the rest of the shell suite plus the Seatbelt spot-checks in
docs/TESTING.md still need a manual hardware run before a release that
advertises macOS.
* **NFS `UNSTABLE`+`COMMIT` is deliberately unimplemented.** Every WRITE is
`FILE_SYNC`-honest and `NFSPROC3_COMMIT` returns `PROC_UNAVAIL`. Implementing
it with imperfect verifier semantics reintroduces the data-loss class
Expand All @@ -247,12 +250,11 @@ Not regressions — deliberate, documented state. Don't "fix" them by accident.
per-phase median regression >5% relative *and* >10ms absolute.
* **Read absolute medians, not ratios.** The native and Vfs legs do not share
controlled page-cache state, so a `vfs / native` ratio moves when the
denominator moves. In the committed baselines, `status` and `diff` ratios
worsened roughly tenfold across two runs while the Vfs times *improved* —
the native leg had simply gotten faster. Quote absolute times with
dispersion; treat a bare ratio as unreviewed. The single-run
`.agents/benchmarks/baseline-*.json` files are profiled single shots, not
the scoreboard.
denominator moves: in the committed baselines the `status` and `diff`
ratios worsened tenfold across two runs while the Vfs times *improved*.
Quote absolute times with dispersion; treat a bare ratio as unreviewed. The
single-run `.agents/benchmarks/baseline-*.json` files are profiled single
shots, not the scoreboard.
* **The workload benchmark refuses to guess.** `.agents/benchmarks/fixtures/`
is gitignored, so the canonical codex fixture is missing on CI and on fresh
clones. `git-workload-benchmark.py` errors rather than silently measuring
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,12 @@ pack/adopt manifest fields and reserved exit statuses remain unchanged.
`revert`, `checkpoint`, and plain `backup` refuse while hollow, and the
streamer never publishes an empty chunk body. `vfs version --json`
advertises `features.adoptRemote`.
- CI now runs the macOS runtime for real: a `macos-runtime` job on
macos-latest executes the NFS + Seatbelt validation script (previously a
manual hardware gate) and both remote-tier suites over the unprivileged
`mount_nfs` path, failing on any suite SKIP. The suites gate per platform;
only the live-checkpoint and streamer legs stay Linux-only, because the
control socket and streamer exist only in the Linux mount owner.

### Fixed

Expand Down
10 changes: 6 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -322,9 +322,11 @@ One Cargo workspace, five crates:

**Linux is first-tier** (FUSE and NFS backends, `vfs run` sandbox, full
validation gate). **macOS is second-tier**: NFS mount plus a sandboxed `vfs
run` (Seatbelt with default-deny read scoping), validated by a manual release
gate on real hardware — see [docs/TESTING.md](docs/TESTING.md). No other
platforms are supported.
run` (Seatbelt with default-deny read scoping). CI exercises the macOS
runtime for real — the NFS mount path, the Seatbelt read-scoping check, and
the remote-tier suites all run on macos-latest — leaving a short list of
Seatbelt spot-checks to real hardware; see
[docs/TESTING.md](docs/TESTING.md). No other platforms are supported.

## Getting started

Expand Down Expand Up @@ -445,7 +447,7 @@ looks fine and is quietly wrong.
- **[User Manual](docs/MANUAL.md)** — complete CLI reference (generation-checked against the binary)
- **[Agent Filesystem Specification](docs/SPEC.md)** — SQLite schema and runtime invariants
- **[Runtime Knobs](docs/KNOBS.md)** — generated ledger of every tunable
- **[Testing](docs/TESTING.md)** — validation gates, benchmark policy, manual macOS release gate
- **[Testing](docs/TESTING.md)** — validation gates, benchmark policy, the macOS runtime gate
- **[AGENTS.md](AGENTS.md)** — working contract for changing this repo
- **[CHANGELOG](CHANGELOG.md)** — fork-era summary
- **[Turso](https://github.com/tursodatabase/turso)** — the in-process SQL database Vfs builds on
Expand Down
27 changes: 17 additions & 10 deletions crates/vfs-cli/tests/test-remote-adopt-e2e.sh
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,9 @@ wait_pid_exit() {
unmount_path() {
local path="$1"
[ -n "$path" ] || return
if command -v fusermount3 >/dev/null 2>&1; then
if [ "$(uname -s)" = "Darwin" ]; then
/sbin/umount "$path" 2>/dev/null || /sbin/umount -f "$path" 2>/dev/null || true
elif command -v fusermount3 >/dev/null 2>&1; then
fusermount3 -uz "$path" 2>/dev/null || true
elif command -v fusermount >/dev/null 2>&1; then
fusermount -u "$path" 2>/dev/null || true
Expand All @@ -73,9 +75,20 @@ stop_live() {
LIVE_LOG=""
}

case "$(uname -s)" in
Linux) ;;
*) skip "requires Linux namespaces and FUSE" ;;
OS_NAME="$(uname -s)"
case "$OS_NAME" in
Linux)
[ -e /dev/fuse ] || skip "requires /dev/fuse for FUSE mounts"
if [ -r /proc/sys/kernel/unprivileged_userns_clone ] &&
[ "$(cat /proc/sys/kernel/unprivileged_userns_clone)" = "0" ]; then
skip "unprivileged user namespaces are disabled"
fi
;;
Darwin)
[ -x /sbin/mount_nfs ] || skip "requires /sbin/mount_nfs for NFS mounts"
[ -x /usr/bin/sandbox-exec ] || skip "requires sandbox-exec"
;;
*) skip "requires Linux (FUSE) or macOS (NFS)" ;;
esac

[ -n "$VFS_BIN" ] || command -v cargo >/dev/null 2>&1 ||
Expand All @@ -85,12 +98,6 @@ command -v git >/dev/null 2>&1 || skip "git is unavailable"
command -v sha256sum >/dev/null 2>&1 || skip "sha256sum is unavailable"
command -v truncate >/dev/null 2>&1 || skip "truncate is unavailable"
[ -x /bin/bash ] || skip "/bin/bash is unavailable"
[ -e /dev/fuse ] || skip "requires /dev/fuse for FUSE mounts"

if [ -r /proc/sys/kernel/unprivileged_userns_clone ] &&
[ "$(cat /proc/sys/kernel/unprivileged_userns_clone)" = "0" ]; then
skip "unprivileged user namespaces are disabled"
fi

if [ -z "$VFS_BIN" ]; then
cargo +nightly build --quiet --manifest-path "$CLI_DIR/Cargo.toml" >/dev/null 2>&1 ||
Expand Down
37 changes: 27 additions & 10 deletions crates/vfs-cli/tests/test-remote-checkpoint-e2e.sh
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,9 @@ wait_pid_exit() {
unmount_path() {
path="$1"
[ -n "$path" ] || return
if command -v fusermount3 >/dev/null 2>&1; then
if [ "$(uname -s)" = "Darwin" ]; then
/sbin/umount "$path" 2>/dev/null || /sbin/umount -f "$path" 2>/dev/null || true
elif command -v fusermount3 >/dev/null 2>&1; then
fusermount3 -uz "$path" 2>/dev/null || true
elif command -v fusermount >/dev/null 2>&1; then
fusermount -u "$path" 2>/dev/null || true
Expand All @@ -65,9 +67,20 @@ stop_live() {
LIVE_SESSION=""
}

case "$(uname -s)" in
Linux) ;;
*) skip "requires Linux namespaces and FUSE" ;;
OS_NAME="$(uname -s)"
case "$OS_NAME" in
Linux)
[ -e /dev/fuse ] || skip "requires /dev/fuse for FUSE mounts"
if [ -r /proc/sys/kernel/unprivileged_userns_clone ] &&
[ "$(cat /proc/sys/kernel/unprivileged_userns_clone)" = "0" ]; then
skip "unprivileged user namespaces are disabled"
fi
;;
Darwin)
[ -x /sbin/mount_nfs ] || skip "requires /sbin/mount_nfs for NFS mounts"
[ -x /usr/bin/sandbox-exec ] || skip "requires sandbox-exec"
;;
*) skip "requires Linux (FUSE) or macOS (NFS)" ;;
esac

[ -n "$VFS_BIN" ] || command -v cargo >/dev/null 2>&1 ||
Expand All @@ -76,12 +89,6 @@ command -v python3 >/dev/null 2>&1 || skip "python3 is unavailable"
command -v git >/dev/null 2>&1 || skip "git is unavailable"
command -v sha256sum >/dev/null 2>&1 || skip "sha256sum is unavailable"
[ -x /bin/bash ] || skip "/bin/bash is unavailable"
[ -e /dev/fuse ] || skip "requires /dev/fuse for FUSE mounts"

if [ -r /proc/sys/kernel/unprivileged_userns_clone ] &&
[ "$(cat /proc/sys/kernel/unprivileged_userns_clone)" = "0" ]; then
skip "unprivileged user namespaces are disabled"
fi

if [ -z "$VFS_BIN" ]; then
cargo +nightly build --quiet --manifest-path "$CLI_DIR/Cargo.toml" >/dev/null 2>&1 ||
Expand Down Expand Up @@ -433,6 +440,12 @@ grep -qi "remote metadata artifact.*chunk bytes are not present" \
fail "writable-open refusal did not identify hollow remote metadata"

# --- Live checkpoint through the control socket ----------------------------
# The session control socket and the background streamer live in the mount
# owner, which serves them only on Linux; on macOS these two legs have no
# product surface to exercise, so they are the one platform-conditional
# region in this suite.
if [ "$OS_NAME" = "Linux" ]; then

LIVE_REMOTE="$TEST_ROOT/remote-live"
mkdir -p "$LIVE_REMOTE"
start_live "$LIVE_ID" "$LIVE_REMOTE" 60000 "$TEST_ROOT/live-owner.log"
Expand Down Expand Up @@ -537,6 +550,10 @@ assert token["uploadedChunks"] + token["reusedChunks"] == token["chunkCount"], t
PY
stop_live

else
echo "note: live-checkpoint and streamer legs run only on Linux (control socket and streamer are Linux-only)"
fi

# --- Journal disabled ------------------------------------------------------
NO_JOURNAL_REMOTE="$TEST_ROOT/remote-no-journal"
mkdir -p "$NO_JOURNAL_REMOTE"
Expand Down
Loading
Loading