Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions INSTALLATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,47 @@ See [Getting Started](docs/getting_started.md#server-path) for a complete TOML
deployment and [`switchyard-server`](crates/switchyard-server/README.md) for the
configuration reference.

## Linux Codex Service

This setup is for single-user Linux machines. It requires a systemd user
session, the Rust toolchain listed above, and Codex CLI 0.134.0 or newer
logged in with a ChatGPT account. Another user's process could take the local
port while the service is stopped and receive your login and prompts.

From a checkout, preview or install the service:

```bash
make install-linux-dry-run
make install-linux
systemctl --user status switchyard
codex -p sy
```

The installer builds and installs `~/.switchyard/bin/switchyard-server`.
It creates `~/.switchyard/composite.toml` if missing and keeps existing edits.
It replaces `~/.config/systemd/user/switchyard.service` and restarts the service.
It writes `~/.codex/sy.config.toml`, backing up a changed profile first.
It leaves shell rc files unchanged. Use `codex -p sy` to select the profile;
`codex login` and other management commands still work as usual.
The profile format requires [Codex 0.134.0 or newer](https://developers.openai.com/codex/config-advanced#profiles).
Remove any old `[profiles.sy]` table from `~/.codex/config.toml` before using it.

Set `SY_HOME` or `SY_PORT` to change the server directory or port (default 4123).
`SY_HOME` must not contain whitespace, control characters, or a trailing
backslash; `SY_PORT` must contain only digits. `XDG_CONFIG_HOME` and `CODEX_HOME`
set the systemd and Codex config directories.

Read server logs with `journalctl --user -u switchyard`. Routing records are
stored in `~/.switchyard/routing.jsonl`. Use `systemctl --user edit switchyard`
for service changes that survive reinstalling.

Remove the service and profile with `make uninstall-linux`. This also removes
marked Codex aliases left by older installs from existing `.bashrc` and
`.zshrc` files. If upgrading an older install, uninstall first and run
`unalias codex` in any open shell. The server directory, routing records,
profile backups, and systemd drop-in files stay in place. Use the same path
overrides when installing and uninstalling.

## Rust Libraries

Add the crates needed by an embedded application:
Expand Down
22 changes: 22 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0

.DEFAULT_GOAL := help
.PHONY: help install-linux install-linux-dry-run uninstall-linux

help:
@echo "install-linux Install the systemd user service and Codex profile"
@echo "install-linux-dry-run Preview installation without changes"
@echo "uninstall-linux Remove the service and Codex profile"

## Install the Switchyard background server as a systemd user service.
install-linux:
Comment thread
elyasmnvidian marked this conversation as resolved.
@scripts/linux/install.sh

## Print what install-linux would do, without changing anything.
install-linux-dry-run:
@scripts/linux/install.sh --dry-run

## Remove the systemd user service, the sy Codex profile, and the codex alias.
uninstall-linux:
@scripts/linux/uninstall.sh
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ releases. Pin the version you integrate.
| `switchyard-libsy` | **Beta** | Routing embedded in your own gateway or harness. You own model calls, credentials, and retries. | Trial integrations. API will change before v1.0. |
| `switchyard-llm-client` | **Alpha** | HTTP model calls and protocol translation alongside libsy. | Experiments and pilots. |
| `switchyard-runner` | **Alpha** | Running configured routes inside another runtime, such as NeMo Relay. | Integration work and supervised pilots. |
| `switchyard-server` | **Demo** | A standalone OpenAI- and Anthropic-compatible proxy. | Demos and evaluation only. Not for production. |
| `switchyard-server` | **Demo** | A standalone OpenAI- and Anthropic-compatible proxy, including a local Codex service. | Demos, evaluation, and personal use on single-user machines. Not for production. |

## Community and license

Expand Down
47 changes: 47 additions & 0 deletions scripts/linux/common.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Paths, markers, and helpers shared by install.sh and uninstall.sh.
# The markers must match between the two, which is why they live here.

SY_HOME="${SY_HOME:-$HOME/.switchyard}"
SY_PORT="${SY_PORT:-4123}"
SERVICE_NAME="switchyard.service"
SYSTEMD_USER_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/systemd/user"
CODEX_DIR="${CODEX_HOME:-$HOME/.codex}"
Comment thread
messiaen marked this conversation as resolved.
# Since Codex 0.134.0, `--profile sy` reads sy.config.toml and fails if
# config.toml still has [profiles.sy].
CODEX_PROFILE_CONFIG="$CODEX_DIR/sy.config.toml"
ALIAS_START="# >>> switchyard codex alias >>>"
ALIAS_END="# <<< switchyard codex alias <<<"

say() { printf '%s\n' "$*"; }
step() { printf '\n==> %s\n' "$*"; }

# Deletes the marked block, inclusive, leaving the rest of the file alone.
strip_block() {
local path="$1" start="$2" end="$3" label="${4:-the switchyard block}"
if [[ ! -f "$path" ]] || ! grep -qF "$start" "$path"; then
Comment thread
elyasmnvidian marked this conversation as resolved.
return 1
fi
if ! grep -qF "$end" "$path"; then
say " $path has no end marker for $label; not editing it" >&2
return 1
fi
if (( DRY_RUN )); then
say " would remove $label from $path"
return 0
fi
local temp
temp="$(mktemp)"
awk -v start="$start" -v end="$end" '
index($0, start) { skipping = 1 }
!skipping { print }
index($0, end) { skipping = 0 }
END { if (skipping) exit 1 }
' "$path" > "$temp"
cat "$temp" > "$path"
rm -f "$temp"
say " removed $label from $path"
return 0
}
216 changes: 216 additions & 0 deletions scripts/linux/install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,216 @@
#!/usr/bin/env bash
# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Installs the Switchyard background server as a systemd --user service and sets
# up a `sy` Codex profile.
#
# Keeps existing composite.toml, replaces the service unit, and backs up
# sy.config.toml before replacing it. Use systemctl --user edit switchyard
# for service changes that survive reinstalling.
# Run with --dry-run to print what would happen.

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"

DRY_RUN=0
case "$#:${1:-}" in
0:) ;;
1:--dry-run) DRY_RUN=1 ;;
*) printf 'Usage: %s [--dry-run]\n' "$0" >&2; exit 2 ;;
esac

# shellcheck source=scripts/linux/common.sh
source "$SCRIPT_DIR/common.sh"

if [[ "$SY_HOME" =~ [[:space:][:cntrl:]] || "$SY_HOME" == *\\ ]]; then
say "SY_HOME must not contain whitespace, control characters, or a trailing backslash." >&2
exit 1
fi
if [[ ! "$SY_PORT" =~ ^[0-9]+$ ]]; then
say "SY_PORT must contain only digits." >&2
exit 1
fi

# Runs a command, or prints it when dry running.
run() {
if (( DRY_RUN )); then
say " would run: $*"
else
"$@"
fi
}

# Writes stdin to a file, leaving an existing file untouched.
write_once() {
local path="$1"
if [[ -f "$path" ]]; then
say " keeping existing $path"
cat >/dev/null
return
fi
if (( DRY_RUN )); then
say " would create $path"
cat >/dev/null
else
mkdir -p "$(dirname "$path")"
cat > "$path"
say " created $path"
fi
}

# Writes stdin to a file, keeping any existing version as a timestamped backup.
write_with_backup() {
local path="$1"
if (( DRY_RUN )); then
[[ -f "$path" ]] && say " would back up $path"
say " would write $path"
cat >/dev/null
return
fi
mkdir -p "$(dirname "$path")"
local incoming
incoming="$(mktemp)"
cat > "$incoming"
if [[ -f "$path" ]] && cmp -s "$incoming" "$path"; then
say " $path is already up to date"
rm -f "$incoming"
return
fi
if [[ -f "$path" ]]; then
local backup
backup="$path.switchyard-backup.$(date +%Y%m%d%H%M%S)"
cp "$path" "$backup"
say " backed up $path to $backup"
fi
cat "$incoming" > "$path"
rm -f "$incoming"
say " wrote $path"
}

# Writes stdin to a file, replacing it. Used only for files this script owns.
write_always() {
local path="$1"
if (( DRY_RUN )); then
say " would write $path"
cat >/dev/null
else
mkdir -p "$(dirname "$path")"
cat > "$path"
say " wrote $path"
fi
}

if [[ "$(uname -s)" != "Linux" ]]; then
say "This installer is for Linux only." >&2
exit 1
fi

step "Building release binary"
run cargo build --release --manifest-path "$REPO_ROOT/Cargo.toml" -p switchyard-server

step "Installing binary into $SY_HOME/bin"
run mkdir -p "$SY_HOME/bin"
run install -m 755 "$REPO_ROOT/target/release/switchyard-server" "$SY_HOME/bin/switchyard-server"

step "Writing server config"
# The composite router from examples/run_codex.sh: Terra classifies each user
# turn and sets the tier, Stage drives the tool loop underneath it.
write_once "$SY_HOME/composite.toml" <<'EOF'
schema_version = 1

[llm_clients.chatgpt_backend]
format = "openai_responses"
base_url = "https://chatgpt.com/backend-api/codex"
forward_auth = true

[targets.capable]
id = "gpt-5.6-sol"
llm_client = "chatgpt_backend"

[targets.efficient]
id = "gpt-5.6-luna"
llm_client = "chatgpt_backend"

# chatgpt.com/backend-api/codex is Codex CLI's own private endpoint, not the
# public OpenAI Responses API. It 400s unless store=false and stream=true are
# set explicitly, and it rejects max_output_tokens outright, so the
# classifier's own token cap has to be dropped before the request goes out.
[targets.terra]
id = "gpt-5.6-terra"
llm_client = "chatgpt_backend"
extra_body = { store = false, stream = true }
omit_body_fields = ["max_output_tokens"]

[routes.switchyard]
id = "switchyard"
type = "composite"

[routes.switchyard.classifier]
target = "terra"
base_threshold = 0.5
classify_trigger = "user_turn"

[routes.switchyard.stage]
capable_target = "capable"
efficient_target = "efficient"
confidence_threshold = 0.5
EOF

step "Validating the server config"
if (( DRY_RUN )); then
say " would run: $SY_HOME/bin/switchyard-server --config $SY_HOME/composite.toml --dry-run"
else
"$SY_HOME/bin/switchyard-server" --config "$SY_HOME/composite.toml" --dry-run
fi

step "Writing the systemd user service"
write_always "$SYSTEMD_USER_DIR/$SERVICE_NAME" <<EOF
[Unit]
Description=Switchyard LLM router server

[Service]
Type=simple
ExecStart=$SY_HOME/bin/switchyard-server --config $SY_HOME/composite.toml --host 127.0.0.1 --port $SY_PORT --routing-log-file $SY_HOME/routing.jsonl
Restart=on-failure
Environment=RUST_LOG=info

[Install]
WantedBy=default.target
EOF

step "Loading the systemd user service"
run systemctl --user daemon-reload
run systemctl --user enable "$SERVICE_NAME"
run systemctl --user restart "$SERVICE_NAME"
run sleep 2
if (( ! DRY_RUN )) && ! systemctl --user is-active --quiet "$SERVICE_NAME"; then
say "Service failed to start. See: journalctl --user -u switchyard" >&2
exit 1
fi

step "Adding the sy Codex profile"
# This profile only changes which router answers. Approval and sandbox
# settings are deliberately left out, so the profile cannot loosen how Codex
# asks before it acts. Set those yourself if you want them.
write_with_backup "$CODEX_PROFILE_CONFIG" <<EOF
model = "switchyard"
model_provider = "sy"

[model_providers.sy]
name = "Switchyard"
base_url = "http://127.0.0.1:$SY_PORT/v1"
wire_api = "responses"
requires_openai_auth = true
EOF

step "Done"
say "Server: http://127.0.0.1:$SY_PORT"
say "Logs: journalctl --user -u switchyard"
say "Manage it with: systemctl --user {status,restart,stop} $SERVICE_NAME"
say "Use it with: codex -p sy (requires Codex CLI 0.134.0 or newer)"
say ""
say "If you want the server running even when you are logged out, run:"
say " loginctl enable-linger \$USER"
Loading
Loading