Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,8 @@ jobs:
oci-sandbox-smoke:
name: Real OCI cgroup and network isolation
runs-on: ubuntu-latest
env:
TINY_HARNESS_REQUIRE_DOCKER_INTEGRATION: "1"
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
Expand Down
15 changes: 14 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,7 @@ ANTHROPIC_API_KEY="sk-ant-..." pytest tests/test_integration.py -v -m integratio

The configured `context_window` reserves `max_tokens` for model output. When input history grows too large, the harness compacts oversized tool outputs and can discard complete older turns, preserving the latest user turn and tool-call/result pairing. This budget currently uses a heuristic estimate, not model-specific tokenizers.

The built-in Shell skill requires human approval by default. In `--no-hitl` mode, Shell additionally requires `--allow-unsafe-shell`. The default `shell_sandbox=off` is **not** OS-isolated; Linux users may opt into `--shell-sandbox bubblewrap` for namespace-isolated execution with per-process resource limits. See [Linux Shell Sandbox](docs/SHELL_SANDBOX.md).
The built-in Shell skill requires human approval by default. In `--no-hitl` mode, Shell additionally requires `--allow-unsafe-shell`. The default `shell_sandbox=off` is **not** OS-isolated. Linux can use [Bubblewrap](docs/SHELL_SANDBOX.md) with per-process limits, or opt into a [Docker OCI sandbox](docs/OCI_SANDBOX.md) with container-wide cgroup limits (validated in real GitHub CI).

## Optional Linux Shell Sandbox

Expand All @@ -323,6 +323,19 @@ tiny-harness --provider deepseek --skills files,shell --shell-sandbox bubblewrap

Requires system-installed `bubblewrap` and `prlimit`, and working unprivileged namespaces. Sandboxed mode blocks external network by default; it never falls back to a host shell if setup fails. These per-process limits are **not** cgroup-wide quotas.

## OCI Docker Sandbox (Linux)

```bash
docker pull alpine:3.20
docker image inspect --format '{{index .RepoDigests 0}}' alpine:3.20
tiny-harness --provider deepseek --skills files,shell \
--shell-sandbox docker \
--sandbox-image 'alpine@sha256:<your_64_hex_digest>' \
--sandbox-cpus 1.0 --sandbox-memory-mb 1024 --sandbox-pids-limit 64
```

Use the **actual** digest returned by `docker image inspect`; the displayed digest is only a placeholder. The Docker sandbox disables external networking, requires a pre-pulled image and uses a container read-only root with a writable workspace. A real OCI CI job verifies cgroup and filesystem/network isolation. Docker daemon privileges and third-party Python tools are still outside this boundary. See [OCI Sandbox details](docs/OCI_SANDBOX.md).

## Security

Tool arguments are checked with JSON Schema Draft 2020-12 and declared filesystem paths are normalized before tool execution. **These checks are not an OS sandbox**. Read [security boundaries](docs/security.md) and the [phased optimization status](docs/OPTIMIZATION_STATUS.md) before deploying to untrusted environments.
Expand Down
108 changes: 108 additions & 0 deletions docs/OCI_SANDBOX.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
# OCI / Docker Shell Sandbox

The built-in async Shell skill has three execution modes:

| `shell_sandbox` | Behavior | Isolation claim |
|---|---|---|
| `off` (default) | Host subprocess, requires HITL by default | **No OS isolation** |
| `bubblewrap` | Linux user/mount/network namespaces, per-process rlimits | Host-dependent; real namespace test may skip in CI |
| `docker` | Linux Docker OCI container, cgroup resource caps, network none | Real Docker integration test required and passing on Ubuntu CI |

The Docker backend applies **only** to the registered async
`run_command` Shell skill. File and third-party Python tools run on the
host and are not sandboxed by this setting.

## Configure a trusted, pre-pulled immutable image

Choose and audit an image suitable for the commands you intend to run.
For a local demonstration with Alpine:

```bash
docker pull alpine:3.20
docker image inspect --format '{{index .RepoDigests 0}}' alpine:3.20
```

Copy the resulting `repository@sha256:...` into `--sandbox-image`.
Digest pinning ensures the selected image cannot silently change during
execution. The agent **never pulls images** (`--pull=never`). Supply
only images you trust; the daemon's image store and Docker access are
trusted host components.

```bash
tiny-harness \
--provider deepseek \
--workspace /absolute/path/to/disposable-project \
--skills files,shell \
--shell-sandbox docker \
--sandbox-image 'alpine@sha256:<your_64_hex_digest>' \
--sandbox-cpus 1.0 \
--sandbox-memory-mb 1024 \
--sandbox-pids-limit 64
```

The image digest in the example is a **placeholder**, not a usable pin.
Set `--sandbox-image` to the actual inspected `repository@sha256`
reference. No-hitl Shell still requires the separate explicit
`--allow-unsafe-shell` opt-in; keep HITL enabled for regular use.

## Runtime behavior

The builder uses a fixed Docker CLI argv and passes the model's command
only as a final argument to `/bin/sh -c` **inside** the container:

- `--network=none` blocks container external networking.
- `--read-only` makes the image root filesystem read-only, with a private
bounded `/tmp` tmpfs.
- `--mount type=bind,...,target=/workspace` exposes exactly the selected
host project as the writable workspace. All cwd paths must remain inside
it, and system root directories are rejected.
- `--cap-drop=ALL` plus `--security-opt=no-new-privileges:true`
removes ambient capabilities and blocks gaining additional privileges.
- `--user uid:gid` uses the non-root host identity (root host user is
rejected) to preserve project file ownership.
- `--memory` and `--memory-swap` impose container memory/swap ceilings;
`--cpus` sets aggregate CPU quota; `--pids-limit` limits container
processes. These are **cgroup** limits across container processes,
unlike Bubblewrap's inherited per-process rlimits.
- Each call receives a generated unique name, a private Docker CLI config
directory, and is cleaned with daemon-side `docker rm -f` even after
timeout or task cancellation. Failures never fall back to host Shell.

The existing Shell output cap and wall-clock timeout still apply.
The Bubblewrap-only `sandbox_cpu_seconds` and `sandbox_file_mb`
flags do not create Docker disk-usage quotas.

## Real validation

GitHub CI job `Real OCI cgroup and network isolation` installs package
dependencies, pulls the test image, resolves an immutable image digest,
then executes the Shell skill **inside a real container**. It asserts:

1. The container's cgroup memory and PID limits match the requested values.
2. The container has no ordinary network routes.
3. The host file outside the selected workspace is not visible.
4. Writing into the designated workspace succeeds.
5. Timed-out and cancelled jobs leave no running/retained named container.

Unlike Bubblewrap's optional smoke, the Docker job is **mandatory** and
fails if the required image environment is missing. Its passing state is
evidence for this tested GitHub runner configuration, not universal proof.

## Trust boundary and remaining risks

**Docker daemon access is highly privileged**. Never mount
`/var/run/docker.sock`, pass daemon credentials to models, or expose the
Docker API to untrusted users. This implementation targets a trusted local
daemon, not hostile multi-tenant compute, rootless Kubernetes, or remote
Docker daemons.

Other risks remain: workspace contents and executable image code must be
trusted appropriately; volume paths could be swapped by concurrent host
processes; Docker daemon and kernel vulnerabilities are out of scope;
system-default seccomp profiles are daemon dependent; the shared host daemon
is not an independent security trust domain.

Production-grade multi-tenant use needs a dedicated daemon/VM per tenant,
audited images, verified seccomp/AppArmor/SELinux profiles, identity mapping
and tighter filesystem protections, policy gates for *every* tool handler,
and integration tests on each supported host and kernel.
3 changes: 2 additions & 1 deletion docs/OPTIMIZATION_STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,13 @@
| Phase 4 | [#22](https://github.com/1998x-stack/tiny-harness/pull/22) | Draft 2020-12 JSON Schema(嵌套/组合/数组),注册时检查 Schema,拒绝远程引用 | PR CI 通过;引入 jsonschema 运行时依赖 |
| Phase 5 | [#24](https://github.com/1998x-stack/tiny-harness/pull/24) | Linux Bubblewrap 可选 shell namespace + workspace-only bind, 缺失时 fail-closed | Python CI 通过,实际 namespace smoke 最初被跳过 |
| Phase 5 | [#25](https://github.com/1998x-stack/tiny-harness/pull/25) | prlimit CPU/虚拟内存/单文件大小逐进程资源约束 | Python CI 通过,非 cgroup 总配额 |
| Phase 6 | [#27](https://github.com/1998x-stack/tiny-harness/pull/27) | OCI/Docker 可选后端,固定镜像摘要、只读根、network=none、cgroup CPU/内存/PID 限制及容器回收 | GitHub Runner 实际 Docker 隔离、cgroup、超时/取消清理测试通过 |

## 待完成事项(不应标记为已修复)

### P1:继续按独立 PR 完成

1. **OS 沙箱验证及增强**:已实现可选 Linux Bubblewrap namespace 和 prlimit;尚需真实机器上的隔离测试、cgroup 总资源配额、系统级加固和 Windows/macOS 后端。默认 off 仍是不隔离的 Shell,**禁止**把 opt-in Bubblewrap 描述为完整生产沙箱。
1. **OS 沙箱验证及增强**:已实现可选 Linux Bubblewrap 与 OCI Docker 后端;Docker 的 cgroup 限制及实际网络/工作区隔离已在 GitHub Linux runner 通过。仍需独立宿主环境的恶意负载评估、seccomp/AppArmor/SELinux 强化、daemon 隔离、竞态修复和其他平台后端。默认 off 仍是不隔离的 Shell,**不能**据 CI 单环境验证声称完整生产级沙箱。
2. **ToolDef 参数级策略**:基础风险枚举与 no-HITL Shell 显式 opt-in 已完成;后续引入细粒度 allow/ask/deny、按命令参数审批策略,扩展其他危险工具的无 HITL 拒绝边界。
3. **Provider 全面契约**:Anthropic 自定义 endpoint、截断判断和首次输出前的有限流式重试已完成;仍需统一非流式/流式重试、DeepSeek thinking 透传、HTTP 客户端连接复用及真实 API 冒烟。
4. **上下文窗口**:已提供可配置窗口与启发式压缩;仍需 provider 真实 usage 回收、tokenizer 精确统计、可审计摘要和更精细的窗口分层治理。
Expand Down
3 changes: 2 additions & 1 deletion docs/security.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,14 @@ tiny-harness 是可执行工具的 Agent harness,**不是安全沙箱**。模
- `ToolDef` 未声明风险时默认 mutation;未知 risk_level 在注册阶段拒绝;safe 风险等级仍执行文件路径守卫。
- `--no-hitl` 下内置 Shell 工具默认不可执行;使用 `--no-hitl --allow-unsafe-shell` 才能显式启用(不等于沙箱)。
- 可选 `shell_sandbox="bubblewrap"` 为 Linux 内置 async Shell 提供 mount/network/PID namespace 约束,启动失败不自动降级;`prlimit` CPU/AS/FSIZE 限制是逐进程限制,非 cgroup 总配额。参见 [SHELL_SANDBOX.md](SHELL_SANDBOX.md)。
- 可选 `shell_sandbox="docker"` 在 Linux 上使用摘要固定的本地 OCI 镜像、非 root UID、只读根、禁网络、最小权限和 cgroup CPU/内存/PID 总限制;真实容器 CI 验证通过,但 Docker daemon、镜像、工作区挂载仍是信任边界。参见 [OCI_SANDBOX.md](OCI_SANDBOX.md)。
- built-in Shell skill 的 async handler 运行于指定 cwd;POSIX 上超时/取消时终止进程组;不继承 `*_API_KEY` 等未在 allow-list 的环境变量。
- 会话快照使用原子替换与受限文件权限(POSIX `0600`)。

## 未提供的隔离能力

- **默认 `shell_sandbox=off` 时 Shell 没有操作系统级文件系统隔离**:即使 cwd 在 workspace,命令仍可能访问机器上该进程有权限访问的路径。开启 bubblewrap 只是选配 Linux namespace 隔离,尚未在所有目标环境做端到端验证。
- Shell 可能访问网络、用户凭据文件、Git 配置及 SSH agent;环境变量白名单不能覆盖全部隐式凭据路径。
- 默认不隔离的 Shell 仍可能访问网络、用户凭据文件、Git 配置及 SSH agent;环境变量白名单不能覆盖全部隐式凭据路径。Docker 模式通过独立容器约束这条路径,但不保护其他 Python 工具。
- 历史同步 `run_command` 辅助函数保留用于兼容,并未获得与 registered async Shell 等价的进程组取消保障。
- Path guard 的检查和 handler 的实际打开文件不是一个不可分割的 OS 操作;工作区内容可被外部进程并发替换时仍有 TOCTOU 风险。
- 在 Windows 上,异步 handler 的 `process.kill()` 不提供 POSIX 进程组等价的孙进程终止保障。
Expand Down
6 changes: 6 additions & 0 deletions tests/test_docker_sandbox.py
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,8 @@ async def test_real_docker_cgroup_network_and_workspace_boundary(tmp_path):
"""Actually execute the container when CI supplies a pulled immutable image."""
image = os.environ.get("TINY_HARNESS_TEST_DOCKER_IMAGE")
if not image:
if os.environ.get("TINY_HARNESS_REQUIRE_DOCKER_INTEGRATION") == "1":
pytest.fail("Required Docker integration image was not configured")
pytest.skip("Docker integration requires TINY_HARNESS_TEST_DOCKER_IMAGE")
workspace = tmp_path / "workspace"
workspace.mkdir()
Expand Down Expand Up @@ -166,6 +168,8 @@ async def test_real_docker_timeout_removes_daemon_container(tmp_path, monkeypatc
"""A timed-out Docker CLI must not orphan its daemon-owned container."""
image = os.environ.get("TINY_HARNESS_TEST_DOCKER_IMAGE")
if not image:
if os.environ.get("TINY_HARNESS_REQUIRE_DOCKER_INTEGRATION") == "1":
pytest.fail("Required Docker integration image was not configured")
pytest.skip("Docker integration requires TINY_HARNESS_TEST_DOCKER_IMAGE")
import secrets
import subprocess
Expand Down Expand Up @@ -195,6 +199,8 @@ async def test_real_docker_cancel_removes_daemon_container(tmp_path, monkeypatch
"""Client-side task cancellation must also force container teardown."""
image = os.environ.get("TINY_HARNESS_TEST_DOCKER_IMAGE")
if not image:
if os.environ.get("TINY_HARNESS_REQUIRE_DOCKER_INTEGRATION") == "1":
pytest.fail("Required Docker integration image was not configured")
pytest.skip("Docker integration requires TINY_HARNESS_TEST_DOCKER_IMAGE")
import asyncio
import secrets
Expand Down
Loading