Skip to content

fix(cache): preserve CacheRuntimeClass template resources when unset - #6165

Open
btxu-db wants to merge 1 commit into
fluid-cloudnative:masterfrom
btxu-db:fix/cacheruntime-preserve-template-resources
Open

fix(cache): preserve CacheRuntimeClass template resources when unset#6165
btxu-db wants to merge 1 commit into
fluid-cloudnative:masterfrom
btxu-db:fix/cacheruntime-preserve-template-resources

Conversation

@btxu-db

@btxu-db btxu-db commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Ⅰ. Describe what this PR does

When a CacheRuntimeClass template declares container resources and the CacheRuntime does not
set spec.master.resources / spec.worker.resources, the template's values were silently
reset to {} on the first reconcile after creation. The AdvancedStatefulSet's generation
went from 1 to 2 and the pods rolled once, with no error and no event — a component the user
capped at 2Gi could then consume the whole node.

Why it happened. syncRuntimeSpec did guard against the zero value, but only when
deciding what to assign to a local variable; the zero value was passed to
SyncComponentSpec anyway:

var workerResources corev1.ResourceRequirements
if runtime.Spec.Worker.Resources.Requests != nil || runtime.Spec.Worker.Resources.Limits != nil {
    workerResources = runtime.Spec.Worker.Resources
}

updateResources treats an empty ResourceRequirements as a valid desired state meaning
"clear the resources". That is deliberate and covered by its own unit test
(sync_component_spec_test.go, "should update to nil resources (remove limits)"), so it
faithfully wrote the empty value through. The information that the user had specified
nothing was lost at the package boundary, because ComponentSpec.Resources is a value type
and cannot distinguish "unset" from "explicitly empty".

Approach. Make ComponentSpec.Resources a *corev1.ResourceRequirements, so nil
means "leave the workload's current resources untouched". This mirrors
ComponentSpec.Replicas, which is already a pointer documented as
(optional, nil means no change) and already nil-checked by SyncComponentSpec:

// 1. Update replicas if specified and changed
if newSpec.Replicas != nil {
    if s.updateReplicas(astsToUpdate, *newSpec.Replicas, logger) {

// 3. Update resources if specified
if newSpec.Resources != nil {
    if s.updateResources(astsToUpdate, *newSpec.Resources, logger) {

updateResources itself is unchanged — a non-nil value is still applied verbatim, so
explicitly clearing resources keeps working and its existing test keeps passing.
ComponentSpec is internal to pkg/ddc/cache/component; no CRD or API type changes.

Ⅱ. Does this pull request fix one issue?

fixes #6161

Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.

syncRuntimeSpec had no direct test coverage, which is how this shipped. Added a
Describe("syncRuntimeSpec") block in pkg/ddc/cache/engine/sync_test.go with three specs,
written at the behaviour level (what the AdvancedStatefulSet looks like after a sync) rather
than against updateResources, so they survive any later refactor of the sync path:

  • CacheRuntime specifies no resources -> master and worker both keep the template's values
  • CacheRuntime specifies master resources -> master is updated, worker is left alone
  • CacheRuntime specifies worker resources -> worker is updated, master is left alone

The last two matter as much as the first. Without them, simply deleting updateResources
would also make the suite pass; their cross-assertions additionally pin down that the two
components do not bleed into each other.

sync_component_spec_test.go is touched only to pass &corev1.ResourceRequirements{...}
where ComponentSpec literals are built; no assertion in that file changed.

Ⅳ. Describe how to verify it

gofmt -l pkg/ddc/cache/                        # no output
go build ./...
go vet ./pkg/ddc/cache/...
go test -gcflags=all=-l ./pkg/ddc/cache/...    # ok
go test ./pkg/ddc/cache/...                    # 228 passed, up from 225 on the base commit

Without -gcflags=all=-l the suite also reports 12 failures in ufs_test.go and one
gomonkey spec in sync_test.go; those need inlining disabled for the patches to take
effect, fail identically on the base commit, and are unrelated to this change.

The new specs were confirmed to be genuine regression tests: checking out only
pkg/ddc/cache/engine/sync_test.go from this branch into a worktree at the base commit —
tests present, fix absent — fails all three with Expected "0" to equal "2Gi". Reverting
the master guard and the worker guard individually each fails a spec too, so neither half of
the change is left uncovered.

On a cluster. kind v0.23.0 / Kubernetes v1.30.0, the manifests from #6161, only the
controller image differs between the two runs. Polling metadata.generation and
spec.template.spec.containers[0].resources on the worker AdvancedStatefulSet, with 2Gi
declared in the CacheRuntimeClass template and nothing in the CacheRuntime:

before

  1s  gen=1 res={"limits":{"memory":"2Gi"}}  dataset=NotBound
 33s  gen=2 res={}                           dataset=Bound
 61s  gen=2 res={}                           dataset=Failed

after

  1s  gen=1 res={"limits":{"memory":"2Gi"}}  dataset=NotBound
  6s  gen=1 res={"limits":{"memory":"2Gi"}}  dataset=Bound
        (unchanged through 90s)

Ⅴ. Special notes for reviews

The Dataset reaching Failed in the "before" run is #6160: the spurious rollout is a
transient runtime outage, and Failed is a one-way trap. This branch does not contain that
fix, yet the Dataset stays Bound after this change — because the spurious rollout no
longer happens. The two issues are still independent: #6160 also triggers on legitimate
rollouts (an image or replica change), so it needs its own fix.

This is the narrow fix for the reported symptom. It does not address a second, distinct way
the same code path loses resources: when a worker uses a processMemory tiered-store level,
handleProcessMemory adds the level's quota on top of the container's memory limit, so the
stored value is baseline + quota while syncRuntimeSpec only knows the baseline and
overwrites the sum away. That reproduces with this PR applied — a CacheRuntime with
worker.resources.limits.memory: 4Gi and processMemory.quota: 8Gi shows
gen=1 mem=12Gi at 1s and gen=2 mem=4Gi at 6s — and cannot be fixed by nil-handling,
since the value the sync path would need does not exist in any single field. Filing that
separately; it likely wants syncRuntimeSpec to compute the desired pod template through
the existing transform chain and diff that, rather than assembling raw spec fields.

Motivation:
When a CacheRuntimeClass template declares container resources and the
CacheRuntime does not set spec.master.resources / spec.worker.resources,
the template values were silently reset to {} on the first reconcile after
creation. The AdvancedStatefulSet's generation bumped from 1 to 2 and the
pods rolled once, with no error or event. A component the user capped at
2Gi could then consume the whole node.

syncRuntimeSpec already guarded against the zero value, but only when
choosing what to assign to a local variable; the zero value was passed on
to SyncComponentSpec regardless. updateResources treats an empty
ResourceRequirements as a valid desired state meaning "clear the
resources" -- a deliberate contract covered by its own unit test -- so it
faithfully wrote the empty value through. The information that the user
had not specified anything was lost at the package boundary, because
ComponentSpec.Resources is a value type and therefore cannot distinguish
"unset" from "explicitly empty".

Approach:
Make ComponentSpec.Resources a *corev1.ResourceRequirements so that nil
means "leave the workload's current resources untouched", mirroring the
existing ComponentSpec.Replicas field, which is already a pointer
documented as "nil means no change". syncRuntimeSpec now yields nil when
the user specified neither requests nor limits, and SyncComponentSpec
skips updateResources on nil, exactly as it already does for Replicas.

updateResources itself is unchanged: a non-nil value is still applied
verbatim, so explicitly clearing resources keeps working and its existing
tests keep passing.

Validation:
- gofmt -l pkg/ddc/cache/ (no output)
- go build ./...
- go vet ./pkg/ddc/cache/...
- go test -gcflags=all=-l ./pkg/ddc/cache/... -> ok
- go test ./pkg/ddc/cache/... -> 228 passed, up from 225 on the base
  commit. Without the flag the suite also reports 12 failures in
  ufs_test.go and one gomonkey spec in sync_test.go; those need inlining
  disabled for the patches to take effect, fail identically on the base
  commit, and are unrelated to this change.
- Confirmed the new specs are genuine regression tests: checking out only
  sync_test.go from this branch into a worktree at the base commit --
  tests present, fix absent -- fails all three with
  Expected "0" to equal "2Gi". Reverting the master guard and the worker
  guard individually each fails a spec too, so neither half is uncovered.

Signed-off-by: btxu-db <btxu-db@outlook.com>
@fluid-e2e-bot

fluid-e2e-bot Bot commented Aug 17, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign ronggu for approval by writing /assign @ronggu in a comment. For more information see:The Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fluid-e2e-bot

fluid-e2e-bot Bot commented Aug 17, 2026

Copy link
Copy Markdown

Hi @btxu-db. Thanks for your PR.

I'm waiting for a fluid-cloudnative member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@codecov

codecov Bot commented Aug 17, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 65.21%. Comparing base (0e24a95) to head (2a4968a).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6165      +/-   ##
==========================================
+ Coverage   65.19%   65.21%   +0.01%     
==========================================
  Files         486      486              
  Lines       34150    34151       +1     
==========================================
+ Hits        22263    22270       +7     
+ Misses      10136    10132       -4     
+ Partials     1751     1749       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@btxu-db btxu-db closed this Aug 17, 2026
@btxu-db btxu-db reopened this Aug 17, 2026
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]container resources in CacheRuntimeClass are silently dropped

1 participant