security(kubescape): the offline capability blocks the scanner's policy-artifact fetch, so every scan aborts

🤖 Generated by the Agentic Engineer

Problem

capabilities.kubescapeOffline: enable is the only thing that sets KS_OFFLINE=true on the posture scanner, and with it set the scanner never fetches its Regolibrary controls/frameworks bundle. Nothing else supplies that bundle: /home/nonroot/.kubescape is an emptyDir with no init container or ConfigMap behind it, so it is empty on every pod start.

Verified live on admin@prod this run — every scan in the current pod's ~7-day log window aborts:

2026-08-08T09:12:16Z  warn   failed to list frameworks   open /home/nonroot/.kubescape/security.json: no such file or directory
2026-08-08T09:12:16Z  error  scanning failed             open /home/nonroot/.kubescape/security.json: ...
2026-08-08T13:15:17Z  error  scanning failed             open /home/nonroot/.kubescape/allcontrols.json: ...

The deployed pod carries both KS_OFFLINE=true and KS_DOWNLOAD_ARTIFACTS=true — contradictory settings, with offline winning.

Fix

Disable the offline capability. Cloud submit is prevented by scanV1.keepLocal, not by that flag, and the existing kubescapeScheduler.requestBody override sets it explicitly.

Evidence gathered against chart 1.40.3 as pinned:

templates/kubescape/deployment.yaml emits KS_OFFLINE only under eq .Values.capabilities.kubescapeOffline "enable".
templates/_common.tpl force-injects keepLocal only in that same branch, so the explicit override must stay — it becomes load-bearing rather than redundant.
clusterData.keepLocal is or (offline) (not serviceDiscovery.enabled); the live rendered config has serviceDiscovery: false, so it stays true either way.
The artifact hosts are already in the egress allow-list (github.com, release-assets.githubusercontent.com) — added for exactly this download.

Acceptance criteria

KS_OFFLINE and KS_DOWNLOAD_ARTIFACTS express one coherent intent
The scanner can obtain its policy artifacts after a pod restart
Cloud submit remains closed (no report POST, no account-ID write)
A scheduled scan completes successfully, evidenced by scan data newer than the change

Part of #2933. The separate fail-loudly criterion is tracked on its own.

🔗 Related PRs

#2316 - fix(kubescape): make ClusterSecurityException posture exceptions take effect [merged]
#2443 - fix(kubescape): keep scheduled posture scans local so results persist [merged]
#2452 - fix(kubescape): bump posture scanner to v4.0.10 so exception CRs apply offline [merged]
#2464 - fix(kubescape): disable continuous posture scans on the offline install [merged]
#2507 - fix(apps): make actual-budget openid schema offline-resolvable [merged]

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

Create Plan

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

🤖 Generated by the Agentic Engineer

Post-merge verification is not yet possible — blocked on the next scheduled scan at 2026-08-09T09:12Z. Recording the live evidence so the next run can close this without re-deriving it.

The fix is live in the cluster. #3025 merged at 21:34:05Z; the operator pod restarted at 21:31:33Z and the kubescape pod logged Starting Kubescape server version 4.0.10 at 21:36:10Z.

But nothing has scanned since. The newest workloadconfigurationscans object in the cluster is dated 2026-06-27T14:50:44Z — six weeks stale, across all 1542 objects. The scan schedule is kubescape-scheduler / 12 9 * * * (daily 09:12Z), and it last fired 2026-08-08T09:12:00Z — roughly twelve hours before the fix rolled out. So no scan has yet exercised the change, and the acceptance criterion here (a scan object newer than the fix, showing the posture scanner actually loading frameworks) cannot be evaluated until 2026-08-09T09:12Z.

Do not mark this verified from the pod restart alone. A running server is not a completed scan, and that conflation is the whole failure mode this issue describes.

Second fail-silent layer, which strengthens #3024. The scheduler job reports succeeded: 1 regardless of whether the scan works, because all it does is POST a trigger and read the reply:

method: POST, url: http://operator:4002/v1/triggerAction … body: {"commands":[{"CommandName":"kubescapeScan",…}]}
response: ok

So a completely dead scanner still produces a green CronJob, a Succeeded Job, and no alert — which is exactly why this went six weeks unnoticed. #3024 proposes failing loudly on a framework-load failure; this evidence says the trigger path needs the same treatment, since response: ok currently means only "the operator accepted the request", never "a scan ran".

Unrelated to this issue but visible while checking: the operator is logging error in SBOMWatch: missing WLID continuously (~every 30s since restart). That is the kubevuln/SBOM path, not posture — filing separately if it is not already tracked.

🤖 Generated by the Agentic Engineer

Correction to my note above: I checked the SBOMWatch: missing WLID errors before filing, and they are noise, not a defect — no separate issue needed. The CVE path they belong to is healthy and producing fresh data right now: 397 vulnerabilitymanifests, 198 summaries and 128 sbomsyfts, newest 2026-08-08T22:03:57Z — i.e. minutes old.

That check turned out to be worth more than the issue would have been, because it isolates the fault cleanly. The same operator drives both surfaces, and only one of them is dead:

Surface	Newest object	State
CVE (`vulnerabilitymanifests`, `sbomsyfts`)	`2026-08-08T22:03:57Z`	alive, minutes old
Posture (`workloadconfigurationscans`)	`2026-06-27T14:50:44Z`	frozen, six weeks

So this is not a dead operator, a broken schedule, or lost cluster access — all three would take the CVE surface down too. It is specific to the posture scan path, which is exactly what this issue claims (the offline capability blocking the policy-artifact fetch). Anyone verifying the fix should compare against the CVE surface as the live control: if posture is still stale after 2026-08-09T09:12Z while CVE keeps refreshing, the fetch is still blocked.

🤖 Generated by the Agentic Engineer

The prescribed fix has already landed — but it did not restore scanning

Re-measured against admin@prod while picking up the sibling child #3024.

capabilities.kubescapeOffline: disable is already in the HelmRelease, and the deployed scanner
carries no KS_OFFLINE at all. So acceptance criteria 1 and 2 are met, and anyone starting from
the Fix section above would be disabling something already disabled.

Criterion 4 is still not met. The newest stored workloadconfigurationscan is 2026-06-27 —
so the offline change did not restore scanning, and posture has been frozen far longer than #2933
recorded. The scan now aborts a third way:

failed to scan. reason: 'chmod /home/nonroot/.kubescape: operation not permitted'

4 attempts, 4 failures, 0 successes in the current scanner pod's log window. That is EPERM — not
the no such file or directory this issue was filed for, and not the EROFS the HelmRelease
comments anticipate for the Submit-mode path.

Full measurement, including the control that rules out the storage layer and an explicit list of what
I could not establish read-only, is on #2933.

Suggested reframing: the artifact-fetch half of this issue is delivered; what remains is the
chmod failure, which is a different cause and may deserve its own issue rather than being absorbed
here. Leaving that call to whoever picks it up rather than unilaterally re-scoping.

🤖 Generated by the Agentic Engineer

Closing — every acceptance criterion is met, verified live against admin@prod on
2026-08-20T22:0xZ rather than from the manifest alone.

The fix landed in 34a2a07e "fix(kubescape): let the posture scanner fetch its policy artifacts
(#3025)" (2026-08-08), which set capabilities.kubescapeOffline: disable. This issue then stayed
open through the verification window and was never closed.

Criterion	Evidence
`KS_OFFLINE` and `KS_DOWNLOAD_ARTIFACTS` express one coherent intent	On the deployed scanner: `KS_DOWNLOAD_ARTIFACTS=true`, and `KS_OFFLINE` is not set at all. The contradiction is gone at runtime, not just in the chart values.
Policy artifacts survive a pod restart	The current scanner pod is a post-fix restart, and its most recent scheduled run evaluated 227 of 243 controls (overall compliance score 83). A run that evaluates 227 controls has loaded its framework — this is the criterion's positive proof, not an inference from config.
Cloud submit remains closed	`ks-cloud-config` carries `keepLocal: true` (with `serviceDiscovery: false`), and the scan writes results locally with no submit attempt and no account-ID generation failure.
A scheduled scan completes, evidenced by scan data newer than the change	`workloadconfigurationscansummaries` newest object 2026-08-20T10:07:15Z — well after the 2026-08-08 fix, and moving daily.

Why the "54 days stale" reading is not this issue

workloadconfigurationscans (the control-level detail surface) is still frozen at
2026-06-27T14:50:44Z. That is a distinct cause with its own issue, #3275 — the scan that this
issue is about runs and completes, and its results reach the summaries surface on schedule.

Worth stating plainly, because the two look identical from a distance and this issue's title invites
the confusion: reading "posture data is stale" and landing here would mean re-fixing an offline
setting that is already correct. That happened to this run — the repair was scoped and nearly built
before the live check showed the cause was already resolved. The residual belongs to #3275, where
the diagnosis now lives.

Detection of the residual is healthy in the meantime: the staleness alert from #3237 is deployed and
reporting it correctly every 6h.

Navigation Menu

Repository navigation

security(kubescape): the offline capability blocks the scanner's policy-artifact fetch, so every scan aborts #3023

Problem

Fix

Acceptance criteria

The prescribed fix has already landed — but it did not restore scanning

Why the "54 days stale" reading is not this issue

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

security(kubescape): the offline capability blocks the scanner's policy-artifact fetch, so every scan aborts #3023

Description

Problem

Fix

Acceptance criteria

Activity

coderabbitai commented on Aug 8, 2026

devantler commented on Aug 8, 2026

devantler commented on Aug 8, 2026

devantler commented on Aug 19, 2026

The prescribed fix has already landed — but it did not restore scanning

devantler commented on Aug 20, 2026

Why the "54 days stale" reading is not this issue

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions