feat(k8s): deliver config.toml via ConfigMap, drop S3 from the k8s deploy path (studio#138) - #139
Merged
Merged
Conversation
…ploy path (studio#138) Brett, testing a k8s deploy: "deploy failed ... NoSuchBucket" — the wizard's k8s path was pushing a bundle.zip to S3 and wiring openab's `hooks.pre_seed` to pull it at boot, same as ECS. That's a deliberate reuse of the S3-backed carrier from an earlier design decision (k8s_driver.rs's old module doc, sub-slice 3c) — but it means every k8s pod needs AWS credentials just to boot, and needs a control-plane S3 bucket to exist at all. Verified directly against openabdev/openab: - every image's Dockerfile.* bakes `CMD ["openab", "run", "-c", "/etc/openab/config.toml"]` — a plain local file path by default. - `load_config_raw_from_source` (openab-core/src/config.rs) falls through to a local file read for any non-http(s)/s3 string. - openab's own Helm chart already mounts config.toml via a ConfigMap at /etc/openab and never overrides the command (charts/openab/templates/configmap.yaml + deployment.yaml) — this isn't a new pattern, it's the upstream project's own existing k8s-native path. ## Change - `studio-cp`: new `provision_config_k8s_configmap` (mirrors the existing `provision_acp_auth_k8s_secret` shape exactly — Patch::Apply, idempotent) server-side-applies config.toml into a k8s ConfigMap and returns a `k8s-configmap://<name>#<key>` ref. `provision_agent_k8s` now calls this instead of pushing a bundle.zip to S3 / injecting a pre_seed hook / resolving a control-plane bucket at all. - `oabctl::secrets`: new `parse_k8s_configmap_uri`, same shape as the existing `parse_k8s_secret_uri`. - `oabctl::k8s_driver::build_deployment`: a `k8s-configmap://` configFrom mounts that ConfigMap at /etc/openab and leaves the container command unset (image's own default CMD reads it) — no override, no S3. A legacy `s3://`/`http(s)://` configFrom (old stored manifests, and the still-untouched compose-library k8s path) keeps the old override-the-command behavior for backward compatibility. - Dropped the S3-stored-manifest create-vs-redeploy lookup for the wizard's k8s path entirely — every field `build_default_k8s_manifest` sets is already resent by the wizard on every call, so there was nothing a stored manifest preserved that this doesn't already provide. `K8sDriver::apply()` already reports Created-vs-Updated from the live Deployment's own existence, no stored manifest needed for that either. The AWS/ECS path and the older compose-library k8s path (`provision_from_library_k8s`) are untouched — this only changes the New Fleet wizard's k8s deploy (`provision_agent_k8s`). Verification: could not compile locally (this sandbox reliably OOMs on aws-sdk-ec2 regardless of which crate is targeted) — field types (ConfigMap/Volume/VolumeMount/ConfigMapVolumeSource) verified by reading the pinned k8s-openapi 0.24 source directly rather than guessing. CI is the real gate here. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2 tasks
brettchien
added a commit
that referenced
this pull request
Aug 29, 2026
…o#138 follow-up) (#140) Brett hit "namespaces \"openab-studio\" not found" applying the ConfigMap right after #139 landed — the New Fleet wizard's "+ Create new namespace…" field (studio#119) only ever *typed* a name into the deploy request; nothing actually created the k8s Namespace object, so picking it 404'd the instant anything (ConfigMap/Secret/Deployment) tried to apply into it. New `ensure_namespace_k8s` (Patch::Apply, same idempotent shape as `provision_acp_auth_k8s_secret` — a no-op if the namespace already exists), called at the top of both k8s provision paths: `provision_agent_k8s` (the wizard, where Brett hit this) and `provision_from_library_k8s` (the older compose-library path — same gap, fixed for consistency). Verification: could not compile locally (this sandbox reliably OOMs on aws-sdk-ec2). Field types (Namespace/ObjectMeta) verified by reading the pinned k8s-openapi 0.24 source directly. CI is the real gate. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brett hit
deploy failed ... NoSuchBuckettesting a k8s deploy through the New Fleet wizard. Root cause: the k8s path was reusing the exact same S3-backedhooks.pre_seedbundle-carrier the ECS path uses (a deliberate earlier design decision — see the module doc this PR replaces ink8s_driver.rs) — meaning a k8s pod needs AWS credentials just to boot, and a control-plane S3 bucket has to exist at all.Verified directly against
openabdev/openabbefore writing any of this:Dockerfile.*bakesCMD ["openab", "run", "-c", "/etc/openab/config.toml"]— a plain local path by default.load_config_raw_from_source(openab-core/src/config.rs) falls through to a local file read for anything that isn'thttp(s):///s3://./etc/openaband never overrides the command (charts/openab/templates/configmap.yaml+deployment.yaml) — not a new pattern, the upstream project's own existing k8s-native path.Change
provision_config_k8s_configmap(mirrorsprovision_acp_auth_k8s_secret's exact shape —Patch::Apply, idempotent) server-side-appliesconfig.tomlinto a k8sConfigMap, returns ak8s-configmap://<name>#<key>ref.provision_agent_k8scalls this instead of pushingbundle.zipto S3 / injecting apre_seedhook / resolving a control-plane bucket at all.parse_k8s_configmap_uri, same shape as the existingparse_k8s_secret_uri.k8s-configmap://configFrommounts that ConfigMap at/etc/openaband leaves the container command unset (image's own default CMD reads it) — no override, no S3. A legacys3:///http(s)://configFrom(old stored manifests, and the still-untouched compose-library k8s path) keeps the old override-the-command behavior.build_default_k8s_manifestsets is already resent by the wizard on every call, so there was nothing a stored manifest preserved that this doesn't already provide.K8sDriver::apply()already reports Created-vs-Updated from the live Deployment's own existence.The AWS/ECS path and the older compose-library k8s path (
provision_from_library_k8s) are untouched — this only changes the New Fleet wizard's k8s deploy (provision_agent_k8s).deploy_provision_agent's public signature (MCP tool + Tauri bridge) is unchanged.Test plan
build_deploymentunit tests: mounts ConfigMap + skips command override fork8s-configmap://, rejects malformed refs, existings3://tests untouched and still pass unchanged (backward-compat path verified by construction, not just claim)parse_k8s_configmap_uriunit tests mirroringparse_k8s_secret_uri'saws-sdk-ec2regardless of which crate is targeted. Field types (ConfigMap/Volume/VolumeMount/ConfigMapVolumeSource) were verified by reading the pinnedk8s-openapi 0.24source directly, not guessed. CI (build-test) is the real gate — please don't merge until it's green.NoSuchBucket, pod boots with zero AWS credentials🤖 Generated with Claude Code