Skip to content

manager/controlapi: reject named generic resource reservations - #3300

Open
breken-ai wants to merge 1 commit into
moby:masterfrom
breken-ai:fix-named-generic-reservation
Open

breken-ai wants to merge 1 commit into
moby:masterfrom
breken-ai:fix-named-generic-reservation

Conversation

@breken-ai

Copy link
Copy Markdown

- What I did

CreateService / UpdateService now reject a service that reserves a named generic resource. Before this change such a service was accepted and could never be scheduled.

validateResources calls genericresource.ValidateTask, but when that check fails it returns nil instead of the error. This has been the case since generic resources were added in 85019da (2017). So a spec such as

"Resources": {"Reservations": {"GenericResources": [{"NamedResourceSpec": {"Kind": "gpu", "Value": "UUID1"}}]}}

passes validation. Tasks can only claim discrete generic resources: genericresource.HasEnough and Claim both return "task should only hold Discrete type". The scheduler's ResourceFilter therefore rejects every node, and the task stays Pending forever with no suitable node (insufficient resources on N nodes). That happens even when the node advertises exactly that resource. The docker CLI already refuses named generic resources for service create/update, but API clients get no error, and the Engine API's GenericResources example shows NamedResourceSpec entries.

- How I did it

validateResources now returns the ValidateTask error as codes.InvalidArgument. This is a one-line change in manager/controlapi/service.go.

- How to test it

  • TestValidateResources and TestValidateResourceRequirements gain a named generic resource case as a bad input, and TestValidateResources also gains a discrete generic resource as a good input. The bad cases fail on master ("An error is expected but got nil") and pass with the fix.
  • go test ./manager/controlapi/ ./manager/scheduler/ ./api/genericresource/ passes. golangci-lint (v2.14.0) reports 0 issues on manager/controlapi and gofmt is clean.
  • End to end: I built dockerd (linux/arm64) from docker-v29.8.1, once as is and once with this change copied into vendor/. Each ran in docker:dind with --node-generic-resource gpu=UUID1, and the service above was posted to /v1.51/services/create:
    • unpatched: HTTP 201, then named.1 Pending ... "no suitable node (insufficient resources on 1 node)", although docker node inspect self lists {"NamedResourceSpec":{"Kind":"gpu","Value":"UUID1"}}
    • patched: HTTP 400 {"message":"rpc error: code = InvalidArgument desc = invalid argument for resource gpu"}

- Description for the changelog

Reject services that reserve named generic resources instead of accepting them and leaving their tasks pending forever.


AI disclosure: an AI coding agent (Claude) found this bug, wrote the fix and test, and drafted this description. The breken-ai account submitted it, and every result above comes from commands that were actually run. The commit carries an Assisted-By: Claude trailer.

validateResources called genericresource.ValidateTask but returned nil
when it failed, so a service reserving a named generic resource (for
example NamedResourceSpec gpu=UUID1) was accepted. Tasks can only claim
discrete generic resources, so the scheduler's ResourceFilter rejects
every node for such a task and it stays pending forever with
"insufficient resources", even on nodes that advertise that resource.

Return the validation error as InvalidArgument.

Assisted-By: Claude
Signed-off-by: breken-ai <312387581+breken-ai@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant