Skip to content

🤖 feat: write a cleanup record before each CoderAIProvider create (#353 part 2a) - #374

Merged
ThomasK33 merged 6 commits into
mainfrom
feat/ai-provider-cleanup-records
Oct 14, 2026
Merged

ThomasK33 merged 6 commits into
mainfrom
feat/ai-provider-cleanup-records

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Oct 14, 2026 •

Copy link
Copy Markdown
Member

Part 2a of #353: the operator now writes a cleanup record before every CoderAIProvider create in Coder. When a namespace deletion (or a gone control plane) releases the provider without a delete, the record stays and names the Coder deployment and the provider ID. The sweep that uses these records is the stacked PR (part 2b). The CoderAIGatewayKey half follows after both.

Refs #353

Design note (option B′, maintainer decision in #353 (comment))

Why records. A namespace deletion removes Coder's pods at once, so no finalizer or hold can delete the provider in Coder (#362 covers only a direct control-plane delete). A sweeper without records would have to guess from absence and could delete another cluster's providers. A record proves which deployment holds which provider.

Record format. One ConfigMap per object, named ck8s-cleanup-<object UID>, label coder.com/cleanup-record=CoderAIProvider, in the operator's namespace: POD_NAMESPACE, else the service account namespace, else coder-system (the namespace of the shipped Role). Data: kind, uid, namespace, name, deploymentID, nonce (the create display name coder-k8s:<uid>), type (the provider type), coderID (once known) and createAttemptTime. The ConfigMap creation time is the created-at.

How this PR meets the invariants of the decision (2b covers 4–8):

# Invariant Where
1 Record before the create. A failed write sends no create. createAIProvider calls writeAIProviderRecord before the create marker. A failed write or a failed deployment-ID read returns before any POST.
2 No secrets in the record. Only the fields above. A test checks the key value is in no record field.
3 Deployment ID from Coder in the same reconcile as the create. coderDeploymentID reads /api/v2/buildinfo deployment_id right before the create, and again when the provider ID is first recorded. Only a canonical UUID is accepted.
5 Exact IDs, nonce. The record keeps the provider ID before the sync step replaces the nonce (ensureAIProviderRecord runs before the status write and before every sync). A forged status.providerID fails checkAIProviderID first, so it never reaches a record.
7 Remove only after Coder confirms. deleteStep removes the record only when deleteInCoder confirms (success or Coder's own 404, or a read that proves nothing of this object exists), and when the status proves no create was ever confirmed. releaseWithoutDelete keeps it.
9 Only the operator writes the records. A namespaced Role manager-role in coder-system (from a kubebuilder marker) with ConfigMap create/update/delete, bound only to the operator service account by the new RoleBinding coder-k8s-cleanup-records. The ClusterRole keeps ConfigMap read only.
10 Provider half first. This PR and 2b touch only CoderAIProvider.

Upgrade. A provider that exists from before this release has no record. The first reconcile that reaches Coder writes one, with the deployment ID that Coder answers then. That includes the disable path for a failing key Secret: the record write there never delays the disable. A malformed spec.baseURL stalls before any Coder access, so such an object gets its record only after the URL is fixed.

Deployment check on delete (review round 2). Before a delete, the controller compares Coder's current deployment ID with the record's. When the control plane now reaches another deployment (for example after its database Secret changed), it sends no delete there and releases the finalizer without a delete. The record stays for a sweep. The record backfill also runs first in every reconcile now, before the base URL and key Secret checks.

Deployment change under a live object (review round 3). When the create path finds a record of another deployment, the object stalls with CoderDeploymentChanged. The record keeps the old locator, and the controller creates nothing in the new deployment.

Reads and deletes (review round 1). Every record read goes through a live API reader (APIReader, wired to mgr.GetAPIReader()), because the cache can miss a record written moments ago. A record delete reads the ConfigMap first, deletes only a valid record of that UID, and uses a UID precondition. When status.providerID is empty, the delete path uses the record's coderID, so a lost status write after a create cannot make the controller treat its own provider as someone else's.

A ConfigMap of the record's name that is not a valid record of that UID stops the reconcile with an error. The operator never writes over it.

Other changes

Validation

Local gates on the tree of this commit before a docs-only edit and a rebase onto main (1c0df24c, #373: CI and E2E script files only; no Go change came with it):

  • Passed: make build-installer, make codegen, make docs-reference, drift (git diff --exit-code), make verify-vendor, make build, make lint, govulncheck, make test, make test-integration, go test -race ./internal/controller/..., make docs-check, actionlint v1.7.10.
  • After the docs edit (one line in docs/explanation/architecture.md) and the rebase: make docs-check, actionlint and go build ./... again.

Mutation checks (each one made a test fail): no record before the create, no provider ID before the sync, record kept after a confirmed delete, record removed on a release without delete, no backfill, no canonical-UUID check of the deployment ID, no name check of the record.

The dogfood of the whole provider half (records plus sweep) is on the stacked PR, because a record alone changes nothing a user can see except the ConfigMap.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

Change-Id: Ice66d8e736e7372a3afd95d8c96a79955aca6b04
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-14T18:58:31.302656Z 76eb17f Manual request
🔒 Security Review ✅ Completed 2026-10-14T18:59:27.610145Z 76eb17f Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 553cad7604

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/controller/coderaiprovider_controller.go
Comment thread internal/controller/coderaiprovider_controller.go
Comment thread internal/controller/cleanup_record.go Outdated
Comment thread internal/controller/coderaiprovider_delete.go Outdated
@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 553cad7604

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

…lete only a valid record, and use the recorded provider ID on delete

Change-Id: I43013d332be2b96542173c7871a73bdf8efe4957
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 07a012629f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 07a012629f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/controller/coderaiprovider_controller.go
Comment thread internal/controller/coderaiprovider_delete.go
Comment thread internal/controller/coderaiprovider_controller.go
…ntrol plane reaches another deployment, and guard the setup

Change-Id: Iee4add631b870c6070066c275f89f84ed86762fa
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 6e1d1bf90f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e1d1bf90f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/controller/coderaiprovider_record.go
Comment thread internal/controller/coderaiprovider_record.go
Comment thread internal/controller/cleanup_record.go
Comment thread internal/app/controllerapp/controllerapp.go
… record the provider type, and default the record namespace to coder-system

Change-Id: I7b699d65437effb2e94cefb175ee758d163b2c83
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 6e1d1bf90f

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e1d1bf90f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/controller/coderaiprovider_delete.go
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 887b0f10db

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 887b0f10db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/controller/coderaiprovider_delete.go Outdated
… an empty status

Change-Id: Ib7fa8097dbaf483d8f50c72676c746ceceabc1c1
Signed-off-by: Thomas Kosiewski <tk@coder.com>
…nup-records

Change-Id: I391ceb4bb8f88ccfa3c5f8a6d3601d9f3b654557
Signed-off-by: Thomas Kosiewski <tk@coder.com>
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76eb17fc39

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/controller/coderaiprovider_record.go
@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 76eb17fc39

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ThomasK33
ThomasK33 added this pull request to the merge queue Oct 14, 2026
Merged via the queue into main with commit 3f1bee7 Oct 14, 2026
13 checks passed
@ThomasK33
ThomasK33 deleted the feat/ai-provider-cleanup-records branch October 14, 2026 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant