diff --git a/README.md b/README.md index c1200c7..bb84e6a 100644 --- a/README.md +++ b/README.md @@ -184,6 +184,72 @@ Ask your AI coding assistant: Merge the "template" remote into the current branch as described in the AGENTS.md file. ``` +## Optional: Security / Pentest Sandbox Setup + +Only relevant once your project has a `docker-compose.yml` with a running app service - skip this section until then. + +This template ships a pentest sandbox: a docker-compose overlay that runs your app with internet egress genuinely blocked (via a firewall sidecar, not just app config), so a dynamic AI pentest (e.g. [Strix](https://strix.ai)) or anything else run against it can't accidentally hit real third-party services (email/SMS/push providers, payment providers, ...). See [peerigon/IT#335](https://github.com/peerigon/IT/issues/335) for the background and process this is part of. + +Files: `docker-compose.pentest.yml`, `docker/apply-firewall.sh`, `docker/Dockerfile.firewall`, `scripts/run-pentest.sh`, `scripts/verify-pentest-network-isolation.sh`, `scripts/network-isolation-check.mjs`. + +
+ +```markdown +# Customize Pentest Sandbox Setup + +First, read this project's actual `docker-compose.yml` and gather: + +1. **Main app/API service name** - the one thing a pentest tool should + test (e.g. `server`, `api`, `backend`). +2. **Its container port** and a **free host port** to publish it on for + this sandbox (must not collide with the normal dev stack). +3. **A cheap, unauthenticated health-check path** that only returns 2xx + once the app (and its DB connection) is actually ready. +4. **Every other service** in docker-compose.yml that publishes a fixed + host port (db, cache, auth provider, mail catcher, ...) - these need + their ports dropped in the overlay so this sandbox can run alongside + the normal dev stack without port conflicts. +5. **Any third-party integration** (email, push, payments, ...) that + needs a local stand-in (e.g. a mail catcher) to keep working without + real network access - wire its env vars into the overlay too. +6. **What the main service's container image can run** - the isolation + check pipes a Node script into the container via `docker compose exec +... node --input-type=module -`. If the image isn't Node-based, adapt + `scripts/verify-pentest-network-isolation.sh` and + `scripts/network-isolation-check.mjs` to use what it does have + (python3, curl, ...). + +## Files to Update + +### docker-compose.pentest.yml + +- Replace every `server` placeholder with the real service name from (1). +- Set the port mapping from (2). +- Add a `: ports: !override []` block for each service from (4). +- Add env vars / local stand-ins from (5). + +### scripts/run-pentest.sh + +- Set `PENTEST_SERVICE`, `PENTEST_HOST_PORT`, `PENTEST_HEALTH_PATH` to + match what you configured above (or leave them as env var overrides for + whoever runs it, whichever you prefer). + +### scripts/network-isolation-check.mjs + +- Add the real third-party API hosts your app talks to, so the check + actually proves those specific integrations can't leak. + +## After Customizing + +Run `./scripts/run-pentest.sh` (needs Docker + a Strix installation - see +[docs.strix.ai](https://docs.strix.ai) - and either `STRIX_LLM`/ +`LLM_API_KEY` env vars or a prior `strix auth` sign-in). It brings up the +sandbox, proves egress is blocked, and only then starts Strix against it; +tears the stack down again on exit either way. +``` + +
+ ## GitHub rulesets (blueprints) The JSON files under [`.github/rulesets/`](./.github/rulesets/) are **blueprints** for GitHub rulesets. GitHub does not apply them from the repository; import or recreate them in your repo’s (or organization’s) ruleset settings. See [`.github/rulesets/README.md`](./.github/rulesets/README.md) for details. diff --git a/docker-compose.pentest.yml b/docker-compose.pentest.yml new file mode 100644 index 0000000..52cdc77 --- /dev/null +++ b/docker-compose.pentest.yml @@ -0,0 +1,64 @@ +# Pentest sandbox overlay - runs your normal docker-compose stack with +# internet egress genuinely blocked at the network-namespace level, so a +# dynamic AI pentest (or anything else run against this stack) can't +# accidentally hit real third-party services. +# +# How it works: `-firewall` shares your main app service's network +# namespace (`network_mode: service:`) and, once, inserts iptables +# OUTPUT rules into that shared namespace: allow loopback/established +# traffic and RFC1918 ranges (so service-to-service calls keep working), +# default-deny everything else. This only affects traffic the app container +# itself originates - inbound traffic via the published port is a different +# iptables path and is unaffected, so you (or a pentest tool) can still +# reach the app normally from the host. See docker/apply-firewall.sh. +# +# We deliberately do NOT use a Docker `internal: true` network for this: +# Docker does not publish ports for a network marked internal, which would +# make the app unreachable from the host too - and adding a second, +# non-internal network alongside an internal one doesn't help either, +# since Docker then routes the container's default outbound traffic +# through the non-internal network, restoring full internet access. Both +# were verified empirically; the namespace-sharing sidecar below is the +# approach that actually works. +# +# ============================== CUSTOMIZE ================================== +# 1. Replace every `server` below with your main app/API service's name - +# the one thing a pentest tool should actually be able to reach and test. +# 2. Give it its own host port (it must not collide with your normal dev +# stack's port for the same service, since this overlay is meant to run +# ALONGSIDE it under a different compose project name - see +# scripts/run-pentest.sh). +# 3. For every OTHER service in your docker-compose.yml that publishes a +# fixed host port (db, cache, auth provider, mail catcher, ...), add a +# block dropping that port here too, or this stack will fail to start +# because it fights your normal dev stack over the same host ports. +# Ask your AI assistant to generate this list from your actual +# docker-compose.yml - example shown below, commented out. +# 4. If your app needs a local stand-in for a third-party integration to +# keep working without real network access (e.g. a mail catcher for an +# email provider), wire its env vars here so the app talks to that +# local stand-in instead of trying the real service. +# ============================================================================= + +services: + server: # <- rename to your main app/API service + ports: !override + - "127.0.0.1:17999:7999" # <- host:container, keep in sync with scripts/run-pentest.sh + + # Repeat for every other service with a fixed host port in your compose + # file, e.g.: + # mongo: + # ports: !override [] + # mailcatcher: + # ports: !override [] + + server-firewall: # <- rename prefix to match your main service if you like + build: + context: ./docker + dockerfile: Dockerfile.firewall + network_mode: "service:server" # <- rename to your main app/API service + cap_add: + - NET_ADMIN + depends_on: + - server # <- rename to your main app/API service + restart: "no" diff --git a/docker/Dockerfile.firewall b/docker/Dockerfile.firewall new file mode 100644 index 0000000..248f8e3 --- /dev/null +++ b/docker/Dockerfile.firewall @@ -0,0 +1,7 @@ +# Tiny image for the pentest-only egress-firewall sidecar (see +# docker-compose.pentest.yml). Not part of the normal app build. +FROM alpine:3.20 +RUN apk add --no-cache iptables +COPY apply-firewall.sh /usr/local/bin/apply-firewall.sh +RUN chmod +x /usr/local/bin/apply-firewall.sh +ENTRYPOINT ["/usr/local/bin/apply-firewall.sh"] diff --git a/docker/apply-firewall.sh b/docker/apply-firewall.sh new file mode 100755 index 0000000..db06c82 --- /dev/null +++ b/docker/apply-firewall.sh @@ -0,0 +1,32 @@ +#!/bin/sh +# Applies an outbound-only firewall to whichever network namespace this +# process runs in. Meant to run in a sidecar with `network_mode: +# service:` and `cap_add: [NET_ADMIN]`, so the rules apply to the +# target container's namespace rather than to a namespace of its own. +# +# Effect: the target container can still be reached from the host (inbound +# traffic via a published port is unaffected - it's a different iptables +# path) and can still talk to other containers on its private network +# ranges (db, cache, auth provider, mail catcher, ...), but any outbound +# connection to a real internet host is dropped, regardless of what env +# vars/API base URLs the app is configured with. +set -eu + +iptables -F OUTPUT + +# Always allow loopback and replies to connections that came in from outside +# (e.g. the host hitting the published port). +iptables -A OUTPUT -o lo -j ACCEPT +iptables -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT + +# Allow traffic to private/container-network address ranges (RFC1918) so +# service-to-service calls within the compose stack keep working. +iptables -A OUTPUT -d 10.0.0.0/8 -j ACCEPT +iptables -A OUTPUT -d 172.16.0.0/12 -j ACCEPT +iptables -A OUTPUT -d 192.168.0.0/16 -j ACCEPT +iptables -A OUTPUT -d 127.0.0.0/8 -j ACCEPT + +# Default-deny everything else outbound (real internet hosts). +iptables -P OUTPUT DROP + +echo "firewall-applied" diff --git a/eslint.config.js b/eslint.config.js index 4b47d31..c997fa2 100644 --- a/eslint.config.js +++ b/eslint.config.js @@ -1,5 +1,22 @@ import typescriptPreset from "@peerigon/configs/eslint/presets/typescript"; +import nodeRules from "@peerigon/configs/eslint/rules/node"; import vitestRules from "@peerigon/configs/eslint/rules/vitest"; import stylesNoDefaultExport from "@peerigon/configs/eslint/styles/no-default-export"; -export default [...typescriptPreset, ...vitestRules, ...stylesNoDefaultExport]; +const nodeGlobals = nodeRules[0]?.languageOptions?.["globals"] ?? {}; + +export default [ + ...typescriptPreset, + ...vitestRules, + ...stylesNoDefaultExport, + { + // Standalone CLI scripts (e.g. the pentest sandbox setup), not app code. + files: ["scripts/**/*.mjs"], + languageOptions: { + globals: nodeGlobals, + }, + rules: { + "unicorn/no-process-exit": "off", + }, + }, +]; diff --git a/scripts/network-isolation-check.mjs b/scripts/network-isolation-check.mjs new file mode 100644 index 0000000..41c14a9 --- /dev/null +++ b/scripts/network-isolation-check.mjs @@ -0,0 +1,65 @@ +// Probes real external hosts from inside a container to verify that the +// pentest sandbox's egress firewall (docker/apply-firewall.sh) actually +// blocks internet access. Exit code 0 = nothing reachable. +// +// CUSTOMIZE FOR YOUR PROJECT: add the real third-party hosts your app talks +// to (email/SMS/push providers, payment providers, analytics, ...) so this +// actually proves *your* integrations can't leak - not just some defaults. + +const targets = [ + // Generic control hosts - always keep at least one of these. + "https://1.1.1.1", + "https://example.com", + + // Add your project's real third-party API hosts here, e.g.: + // 'https://api.brevo.com', + // 'https://exp.host', +]; + +const TIMEOUT_MS = 4000; + +/** @param {unknown} error */ +function describeError(error) { + if (error instanceof Error) { + const cause = error.cause; + + if (cause && typeof cause === "object" && "code" in cause) { + return String(cause.code); + } + if ("code" in error && typeof error.code === "string") { + return error.code; + } + + return error.name; + } + + return String(error); +} + +/** @param {string} url */ +async function probe(url) { + const controller = new AbortController(); + const timer = setTimeout(() => { + controller.abort(); + }, TIMEOUT_MS); + + try { + await fetch(url, { signal: controller.signal }); + console.log(`REACHABLE ${url}`); + return true; + } catch (error) { + console.log(`BLOCKED ${url} (${describeError(error)})`); + return false; + } finally { + clearTimeout(timer); + } +} + +const results = await Promise.all(targets.map((url) => probe(url))); +const reachableCount = results.filter(Boolean).length; +const blockedCount = results.length - reachableCount; + +console.log( + `\nsummary: ${String(blockedCount)}/${String(targets.length)} blocked, ${String(reachableCount)}/${String(targets.length)} reachable`, +); +process.exit(reachableCount > 0 ? 1 : 0); diff --git a/scripts/run-pentest.sh b/scripts/run-pentest.sh new file mode 100755 index 0000000..7fa94ba --- /dev/null +++ b/scripts/run-pentest.sh @@ -0,0 +1,114 @@ +#!/usr/bin/env bash +# Runs a dynamic Strix scan against this project's pentest sandbox stack - +# but ONLY after proving its egress firewall actually blocks internet +# access. If that cannot be confirmed, Strix is never started. +# +# Order of operations (each step gates the next): +# 1. Tear down any leftover pentest stack from a previous run. +# 2. Bring up the full stack plus the firewall sidecar (see +# docker-compose.pentest.yml / docker/apply-firewall.sh for how and why). +# 3. Wait for the firewall sidecar to report it applied its rules. +# 4. Gate: verify no real external host is reachable from the live app +# container. If this fails, abort and tear down - Strix never starts. +# 5. Wait for the app to become reachable on its published port. +# 6. Only now: run Strix against it. +# 7. Tear the stack down on exit (success, failure, or Ctrl-C), unless +# KEEP_STACK=1. +# +# Usage: +# ./scripts/run-pentest.sh +# ./scripts/run-pentest.sh --instruction "..." +# KEEP_STACK=1 ./scripts/run-pentest.sh # leave the stack up afterwards + +set -euo pipefail +cd "$(dirname "$0")/.." + +# ============================== CUSTOMIZE =================================== +# Fill these in for your project (see docker-compose.pentest.yml too). +PENTEST_SERVICE="${PENTEST_SERVICE:-server}" # your main app/API service's compose name +PENTEST_HOST_PORT="${PENTEST_HOST_PORT:-17999}" # host port this overlay publishes it on +PENTEST_HEALTH_PATH="${PENTEST_HEALTH_PATH:-/}" # cheap endpoint that 2xx's once the app+DB are ready +PENTEST_CODE_TARGET="${PENTEST_CODE_TARGET:-.}" # local path Strix should also read as source +PENTEST_PROJECT="${PENTEST_PROJECT:-$(basename "$(pwd)")-pentest}" +# ============================================================================= + +STRIX_SCAN_MODE="${STRIX_SCAN_MODE:-deep}" +STRIX_MAX_BUDGET="${STRIX_MAX_BUDGET:-20}" +TARGET_URL="http://localhost:${PENTEST_HOST_PORT}" + +COMPOSE=(docker compose -p "$PENTEST_PROJECT" -f docker-compose.yml -f docker-compose.pentest.yml) +STACK_UP=0 + +cleanup() { + local exit_code=$? + if [[ "$STACK_UP" == "1" && "${KEEP_STACK:-0}" != "1" ]]; then + echo "==> Tearing down pentest stack ($PENTEST_PROJECT)..." + "${COMPOSE[@]}" down -v --remove-orphans >/dev/null 2>&1 || true + fi + exit $exit_code +} +trap cleanup EXIT + +echo "==> Preflight checks..." +command -v docker >/dev/null || { echo "FAIL: docker not found" >&2; exit 1; } +command -v strix >/dev/null || { echo "FAIL: strix CLI not found (see docs.strix.ai)" >&2; exit 1; } + +# Strix can be configured either via STRIX_LLM/LLM_API_KEY env vars (BYO key) +# or via `strix auth` (model-subscription sign-in, persisted under +# ~/.strix/). Only fail here if NEITHER is present. +if [[ -z "${STRIX_LLM:-}" && -z "${LLM_API_KEY:-}" ]]; then + if [[ ! -f "$HOME/.strix/cli-config.json" && ! -f "$HOME/.strix/subscription-auth.json" ]]; then + echo "FAIL: no STRIX_LLM/LLM_API_KEY env vars and no ~/.strix config found. Run 'strix auth' or set STRIX_LLM/LLM_API_KEY." >&2 + exit 1 + fi + echo "==> No STRIX_LLM/LLM_API_KEY env override - using existing ~/.strix configuration (e.g. 'strix auth' sign-in)." +fi + +echo "==> Clearing any leftover pentest stack from a previous run..." +"${COMPOSE[@]}" down -v --remove-orphans >/dev/null 2>&1 || true + +echo "==> Bringing up the pentest stack (app + egress-firewall sidecar)..." +"${COMPOSE[@]}" up -d --build +STACK_UP=1 + +echo "==> Waiting for the egress-firewall sidecar to apply its rules..." +FW_ATTEMPTS=30 +until "${COMPOSE[@]}" logs "${PENTEST_SERVICE}-firewall" 2>/dev/null | grep -q "firewall-applied" || [[ $FW_ATTEMPTS -eq 0 ]]; do + FW_ATTEMPTS=$((FW_ATTEMPTS - 1)) + sleep 1 +done +if [[ $FW_ATTEMPTS -eq 0 ]]; then + echo "ABORT: ${PENTEST_SERVICE}-firewall sidecar never reported success. Strix will NOT be started." >&2 + "${COMPOSE[@]}" logs "${PENTEST_SERVICE}-firewall" >&2 || true + exit 1 +fi + +echo "==> Gate: verifying the live ${PENTEST_SERVICE} container cannot reach the internet..." +if ! PENTEST_SERVICE="$PENTEST_SERVICE" PENTEST_PROJECT="$PENTEST_PROJECT" ./scripts/verify-pentest-network-isolation.sh; then + echo "ABORT: network isolation could not be verified. Strix will NOT be started." >&2 + exit 1 +fi + +echo "==> Gate passed. Waiting for the app to become reachable on ${TARGET_URL}${PENTEST_HEALTH_PATH}..." +HEALTH_URL="${TARGET_URL}${PENTEST_HEALTH_PATH}" +ATTEMPTS=150 +until curl -fsS "${HEALTH_URL}" >/dev/null 2>&1 || [[ $ATTEMPTS -eq 0 ]]; do + ATTEMPTS=$((ATTEMPTS - 1)) + printf '.' + sleep 2 +done +echo +if [[ $ATTEMPTS -eq 0 ]]; then + echo "ABORT: app never became reachable on ${HEALTH_URL}. Strix will NOT be started." >&2 + echo "---- ${PENTEST_SERVICE} logs (last 100 lines) ----" >&2 + "${COMPOSE[@]}" logs --tail=100 "$PENTEST_SERVICE" >&2 || true + exit 1 +fi + +echo "==> Isolation confirmed and app is ready — starting Strix..." +strix -n \ + -t "$PENTEST_CODE_TARGET" \ + -t "$TARGET_URL" \ + --scan-mode "$STRIX_SCAN_MODE" \ + --max-budget "$STRIX_MAX_BUDGET" \ + "$@" diff --git a/scripts/verify-pentest-network-isolation.sh b/scripts/verify-pentest-network-isolation.sh new file mode 100755 index 0000000..6044228 --- /dev/null +++ b/scripts/verify-pentest-network-isolation.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +# Verifies that the pentest stack's egress firewall (docker-compose.pentest.yml +# + docker/apply-firewall.sh) actually blocks internet access for the *live* +# app container - instead of just trusting an LLM instruction. +# +# Must be run after the stack (including the firewall sidecar) is already +# up - scripts/run-pentest.sh does this automatically as a gate before +# starting Strix. Can also be run standalone for a quick sanity check. +# +# CUSTOMIZE FOR YOUR PROJECT: the probe below runs scripts/network-isolation- +# check.mjs via `node` INSIDE your app container, because that's what +# konsens uses. If your main service's image isn't Node-based, swap the +# `exec ... node --input-type=module -` line for whatever your image does +# have (python3, or a curl/wget one-liner against a couple of real hosts +# plus one that must stay reachable, like your db). +# +# Usage: ./scripts/verify-pentest-network-isolation.sh +set -euo pipefail +cd "$(dirname "$0")/.." + +PENTEST_SERVICE="${PENTEST_SERVICE:-server}" +PENTEST_PROJECT="${PENTEST_PROJECT:-$(basename "$(pwd)")-pentest}" +COMPOSE=(docker compose -p "$PENTEST_PROJECT" -f docker-compose.yml -f docker-compose.pentest.yml) + +echo "==> Checking that the egress-firewall sidecar actually applied its rules..." +if ! "${COMPOSE[@]}" logs "${PENTEST_SERVICE}-firewall" 2>/dev/null | grep -q "firewall-applied"; then + echo "FAIL: ${PENTEST_SERVICE}-firewall sidecar did not report 'firewall-applied'. Is the stack up?" >&2 + exit 1 +fi + +echo "==> Probing outbound connectivity from the live '${PENTEST_SERVICE}' container..." +set +e +OUTPUT=$(cat scripts/network-isolation-check.mjs | "${COMPOSE[@]}" exec -T "$PENTEST_SERVICE" node --input-type=module - 2>&1) +STATUS=$? +set -e +echo "$OUTPUT" + +if [[ $STATUS -ne 0 ]]; then + echo "FAIL: the live ${PENTEST_SERVICE} container can reach the internet." >&2 + exit 1 +fi + +echo "==> PASS: no external host was reachable from the live ${PENTEST_SERVICE} container."