CoderWorkspacePool: prebuilt workspaces with a scale subresource #153

Activity

1. ThomasK33

   added

   enhancementA proposed feature or improvementA proposed feature or improvement

   on Oct 2, 2026
2. ThomasK33

   self-assigned this

   on Oct 2, 2026
3. ThomasK33 commented on Oct 3, 2026

   

   ThomasK33

   on Oct 3, 2026

   MemberAuthor

   More actions

   Feasibility gate result: blocked upstream

   I will not build a scale API for prebuilds yet. Coder has no API that resizes a prebuild pool in place.

   Findings

   I read the Coder source at v2.35.8, v2.36.6, v2.37.3 and main (cc0c868c55b2). I did not test runtime behavior on a licensed deployment.

   1. The count changes only with a new template version. Coder writes a preset's desired_instances only when it imports a template version (coderd/provisionerdserver/provisionerdserver.go:2781 on main). No HTTP handler updates it. The prebuild write endpoints are the deployment-wide pause (PUT /api/v2/prebuilds/settings) and preset invalidation (POST /api/v2/templates/{template}/prebuilds/invalidate). Neither changes the count.
   2. A new version per resize rebuilds the pool. Coder keeps prebuilds only for the active template version. When a new version becomes active, the reconciler deletes every prebuild of the old version and builds the new count (coderd/prebuilds/preset_snapshot.go:337 and :399). Old-version prebuilds cannot be claimed after the promotion (coderd/database/queries/prebuilds.sql:28). Each resize also:
      • marks workspaces built from the previous version as outdated,
      • adds a template version and two audit entries,
      • runs an import job on the provisioners,
      • competes with GitOps re-applies of CoderTemplate and with codertemplates/promote.
   3. The REST API has no per-preset prebuild status. Coder's Prometheus metrics (coderd_prebuilt_workspaces_*) have per-preset desired, running and eligible gauges. The workspace API (owner:prebuilds) can rebuild running, agent-ready and in-progress counts. It cannot show schedule-aware desired counts, expiry, backoff or the failure hard limit.
   4. License detection is simple. GET /api/v2/entitlements returns the feature workspace_prebuilds. The feature is usable when enabled is true and the entitlement is entitled or grace_period.

   What works today

   • Calendar-based pool sizes: add scheduling to the prebuilds block of coder_workspace_preset (Coder v2.24.0 or later, provider v2.6.0 or later). Coder changes the count by hour, weekday and month without a new version. A change to the schedule needs a new version.
   • Static pool sizes from Git: set instances in the template files of a CoderTemplate. Each change publishes a version and rebuilds the pool, like any other template change.
   • Observation: enable Coder's Prometheus endpoint (CODER_PROMETHEUS_ENABLE) and read the coderd_prebuilt_workspaces_* metrics.

   What will unblock this issue

   An upstream API that sets a per-preset count override without a new template version, and that returns per-preset status. A sketch (not filed upstream):

   • PATCH /api/v2/templates/{template}/prebuilds/presets/{preset} (preset ID) with {"desired_instances_override": 6}. null clears it. The override wins over the Terraform count and schedules until it is cleared. Coder refuses presets without a prebuilds block and presets that are not on the active version. The override belongs to one template version, so a client re-applies it after a promotion.
   • GET /api/v2/templates/{template}/prebuilds/presets with effective desired, running, eligible, in progress and preset health.

   Related upstream issues: coder/coder#20549 (per-template prebuild control) and coder/coder#19927 (prebuild counts through template variables).

   Decision

   • I will not build CoderWorkspacePool with a scale subresource until a Coder release has such an API.
   • I will not build a read-only status view now. It cannot show schedule-aware desired counts or why a pool does not fill. The Prometheus metrics already show Coder's own numbers.
   • A smaller exact item stays possible: show each version's preset definitions (names, parameters, configured base instances) on CoderTemplateVersion. I will open it only on user demand.
   • Recheck trigger: a Coder release that adds a runtime count API for prebuild presets.

   Source lines at each tag

   Item	v2.35.8	v2.36.6	v2.37.3	main
   Import writes DesiredInstances (provisionerdserver.go)	2757	2773	2940	2781
   Invalidate route (enterprise/coderd/coderd.go)	580	604	628	647
   Prebuild settings routes (enterprise/coderd/coderd.go)	662-663	703-704	727-728	746-747
   isActive / handleInactiveTemplateVersion (preset_snapshot.go)	337 / 399	337 / 399	337 / 399	337 / 399
   Claim needs the active version (prebuilds.sql)	28	28	28	28
   FeatureWorkspacePrebuilds (codersdk/deployment.go)	192	192	192	194

   ---

   Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: medium
4. ThomasK33

   added 2 commits that reference this issue on Oct 13, 2026

   feat(api): add the dormant CoderWorkspacePool API types

   ...

   d995316

   feat(api): add the dormant CoderWorkspacePool API types

   ...

   c508335
5. github-merge-queue

   added a commit that references this issue on Oct 14, 2026

   🤖 feat: create CoderWorkspacePool members under the three caps, with …

   ...

   21d3ff5
6. ThomasK33 commented on Oct 14, 2026

   

   ThomasK33

   on Oct 14, 2026

   MemberAuthor

   More actions

   Follow-ups for pool PR 12 (activation gate), collected from the final assessments of #359, #364, #379 and #380. The pool kind is dormant, so none of them blocks a merge today. Each one must be fixed or explicitly accepted before activation.

   1. The member cap check is exact only while member reconciles run one at a time. Set MaxConcurrentReconciles=1 for the pool and member controllers explicitly, or make the check race-free.
   2. The status write in deleteFailedMembers does not store observedGeneration. The status shows a stale value until the final write of the reconcile.
   3. A member that becomes used while its pool is deleted is garbage-collected before the pool releases it. The workspace stays in Coder, but the Kubernetes object goes.
   4. Consent-loss deletes act on a cached read in the resolve step. A stale cache can delete warm members once more than needed. No workspace leaks.
   5. Decide whether failures during a Coder outage count toward the failure limit. Today, failed members are deleted without counting while the pool's inputs do not resolve.
   6. Watch CoderControlPlane from the pool controller.
   7. parametersHash is unsalted.
   8. checkCreate 404 handling (CoderWorkspacePoolMember checkCreate: trust only Coder's own 404 on the name lookup (before PR 12) #355), the CoderWorkspacePoolMember: recognize a template-wide last_used_at write in a one-member pool #349 rule, the bindExtendDeadline bound, the OwnershipUnknown control plane check, and the flapping agent timer.

   ---

   Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: medium
7. ThomasK33 commented on Oct 15, 2026

   

   ThomasK33

   on Oct 15, 2026

   MemberAuthor

   More actions

   More follow-ups for pool PR 12, from the final assessment of #387 and #388 (claims). The pool kind is dormant, so none of them blocks a merge today. Each one must be fixed or explicitly accepted before activation.

   1. Review comment 4277607232: two pool reconciles that run at the same time can bind two members to one claim. This needs two operator replicas at once. Planned fix: reserve the chosen member in the claim status before the member update.
   2. Review comment 4277607237: a new pool that reuses a deleted pool's name with another owner overwrites the claim's ownerName. Planned fix: keep the stored name.
   3. Add a CoderWorkspaceClaim watch that maps each claim to its pool. Without it, a new claim does not wake a Ready pool.
   4. The pool's member watch must map bound members by the pool label or spec.poolName. After a bind, their controller reference points at the claim.
   5. A claimant who renames a bound workspace makes the member fail with OwnershipUnknown. Decide whether to accept a rename.
   6. Claims RBAC, including coderworkspaceclaims/finalizers, because the bind sets blockOwnerDeletion.
   7. Add a test where a bound member gets a 404 or a 5xx from Coder and its claim stays Bound.

   ---

   Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: medium