Skip to content

feat: start without all capabilities and fail fast for the actions needing a missing one - #264

Draft
achoimet wants to merge 9 commits into
mainfrom
feat/degrade-on-missing-capabilities
Draft

achoimet wants to merge 9 commits into
mainfrom
feat/degrade-on-missing-capabilities

Conversation

@achoimet

@achoimet achoimet commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Why

The extension binary carries its file capabilities with the effective bit (setcap …+eip). The kernel then refuses to exec it as soon as one of them is not granted to the container (operation not permitted), so a cluster whose policy forbids a single capability — e.g. NET_ADMIN by a Kyverno/Gatekeeper rule — gets a crash-looping extension instead of one where only the attacks needing that capability are unavailable. The chart even warns about it for SYS_RESOURCE.

What

  • setcap …+ip (no effective bit): the binary starts with whatever the container grants.
  • At startup, the permitted capabilities are raised to effective on all threads (extruntime.RaiseCapabilities, from feat(extruntime): raise capabilities at startup and report missing ones extension-kit#181), and the missing ones are logged.
  • Fail fast: each action family checks the capabilities it needs when prepared and fails with a message naming the missing ones, e.g. "Network attacks need the capabilities NET_ADMIN, which the extension does not have. Add them to the capabilities of the extension's container securityContext." The other actions keep working. Families: every attack running a sidecar needs the sidecar base (SETUID, SETGID, SYS_ADMIN, SYS_CHROOT, SYS_PTRACE, DAC_OVERRIDE, NET_ADMIN — the sidecar brings up the loopback of its network namespace); network faults add NET_RAW, DNS error injection BPF, time travel NET_RAW and SYS_TIME. Shutdown needs SYS_BOOT, stop process KILL, SETUID and SETGID, CPU speed DAC_OVERRIDE.
  • The chart's comment about removing capabilities is updated; chart patch version bumped.

The check uses the bounding set: it is what the container's securityContext grants, and what the root helpers the extension spawns (runc, crun, nsenter, tc, …) inherit through RootCommandContext.

Tests

  • Unit tests for the requirement checks and the fail-fast in Prepare.
  • E2E, new case "starts without a capability and fails fast for the attacks needing it": the extension is reinstalled without SYS_TIME → it becomes ready (on main, helm --wait times out on the crash loop), a time travel fails at prepare naming SYS_TIME, a CPU stress still runs, sidecars are cleaned up; then the default config is restored.

The Linux behaviour of +eip vs +ip with a missing capability is shown in steadybit/extension-kit#181.

Depends on

steadybit/extension-kit#181 — go.mod points to its commit (pseudo-version) until extension-kit is released; to be bumped before merging.

steadybit/action-kit#512 — action-kit checked CAP_SYS_PTRACE at package initialization, before RaiseCapabilities runs, so with +ip every attack took the slow namespace lookup and logged a false warning (7 times in this PR's E2E logs vs once on main). action_kit_commons points to that PR's commit; the pseudo-version reads v1.12.1-0… because the v1.13.0 release commit is not on action-kit's main, but main contains everything in v1.13.0 (the release commit only adds changelog lines). To be bumped to a release before merging.

…eding a missing one

The extension binary carried its file capabilities with the effective bit
(setcap ...+eip). The kernel then refuses to exec it as soon as one of them is
not granted to the container, so a cluster forbidding a single capability
(e.g. NET_ADMIN by policy) got a crash-looping extension.

- setcap ...+ip: the binary starts with what the container grants
- raise the permitted capabilities to effective at startup, on all threads
  (extension-kit extruntime.RaiseCapabilities), and log the missing ones
- the actions check the capabilities they need when prepared and fail with
  a message naming the missing ones; the others keep working
- e2e: reinstalled without NET_ADMIN, the extension becomes ready, network
  attacks fail fast naming NET_ADMIN, stress attacks still work

Uses extension-kit from steadybit/extension-kit#181 (pseudo-version) until it is released.
@claude

claude Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Claude finished @achoimet's task in 2m 32s —— View job


Claude's review

  • Check previously unresolved review threads for fixes (none existed)
  • Gather context (read changed files)
  • Review new changes for bugs, security, performance, concurrency
  • Post findings

No unresolved review threads from a prior run existed on this PR, so there was nothing to reconcile.

Summary: I reviewed exthost/capabilities.go, all the Prepare() call-sites, main.go, the Dockerfile/chart changes, and the new unit/e2e tests. I didn't find any bugs, security issues, or concurrency problems.

A few things I specifically verified because they looked like they could be off, but checked out fine:

  • Every action family that shares a common struct (stressAction, networkAction) gets the capability check exactly once via that shared Prepare, so all the concrete actions registered in main.go (stress-cpu/io/memory, all network variants, fill-disk, fill-mem) are covered — no action was missed.
  • requireHeldCapabilities vs requireCapabilities is applied correctly: it's used only for capabilities exercised directly by the extension's own process (SYS_TIME for time travel, SYS_BOOT for shutdown, DAC_OVERRIDE for CPU speed), which matches the three capabilities actually carried as file capabilities in the Dockerfile's setcap call (so those are the only ones RaiseCapabilities can ever make effective) — the bounding-set-only capabilities (NET_ADMIN, NET_RAW, SETUID, etc.) are the ones consumed by root helpers (runc/crun/tc/nsenter) rather than the extension process itself, so requireCapabilities is correct there.
  • Global mutable test doubles missingCapabilities/missingHeldCapabilities in capabilities_test.go are safe since no test in the package uses t.Parallel().
  • action_network_tcp_reset.go's Prepare checks capabilities once itself and then delegates to a.tcp.Prepare/a.l7.Prepare, which each also check — redundant but harmless (early-exit, no observable difference).
  • The new e2e test's capability list passed via Reconfigure exactly matches values.yaml's default add list minus SYS_TIME, and require.NoError(t, err, ...) on the stress action's RunAction call halts the test via t.FailNow() before the later stress.Cancel() could dereference a nil result — no nil-pointer risk there.
  • go.mod/go.sum point to pseudo-versions of action_kit_commons and extension-kit; the PR description already calls out that these need to be bumped to released versions before merging, so I didn't flag it separately.

I was unable to run go build/go test in this sandbox (shell commands that touch the Go toolchain required approval I couldn't grant), so this review is based on static reading of the diff rather than a green build/test run.
· Branch: feat/degrade-on-missing-capabilities

…ability only one family needs

The e2e run showed that a stress sidecar fails without NET_ADMIN (ioctl
SIOCSIFFLAGS: the sidecar brings up the loopback interface of its network
namespace), so NET_ADMIN belongs to the capabilities of every sidecar. The
degraded-mode e2e case now removes a capability used by a single family.
…ime travel needs the network only to block NTP

- shutdown (reboot syscall), time travel (clock change) and CPU speed
  (cpufreq writes) run in the extension's process, where root helpers cannot
  help: they now check the effective set (extruntime.MissingHeldCapabilities),
  which also catches capabilities RaiseCapabilities could not raise
- time travel requires the network capabilities only with disableNtp, the
  clock change needs SYS_TIME alone
Prepare now checks the capabilities of the process, which a test process
does not have on Linux; the tests of other behaviour must not depend on the
machine they run on. The capability tests stub their own scenarios.
@sonarqubecloud

Copy link
Copy Markdown

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant