fix: support PVC overlays for DM built-in volumes - #7024
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe change permits named overlays for built-in data and relay volume claims. DM and DMWorker PVC reconciliation applies these overlays to primary PVCs and excludes them from additional PVC generation. Validation tests cover valid built-in claims, invalid unknown claims, and CEL static costs. ChangesVolume claim overlays
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant DMResource
participant DMPVCNewer
participant PVCOverlay
participant GeneratedPVCs
DMResource->>DMPVCNewer: Reconcile PVCs
DMPVCNewer->>PVCOverlay: Find data volume overlay
PVCOverlay-->>DMPVCNewer: Return matching overlay
DMPVCNewer->>GeneratedPVCs: Apply overlay to data PVC
DMPVCNewer->>GeneratedPVCs: Generate additional PVCs without data overlay
sequenceDiagram
participant DMWorkerResource
participant DMWorkerPVCNewer
participant PVCOverlay
participant GeneratedPVCs
DMWorkerResource->>DMWorkerPVCNewer: Reconcile PVCs
DMWorkerPVCNewer->>PVCOverlay: Find relay volume overlay
PVCOverlay-->>DMWorkerPVCNewer: Return matching overlay
DMWorkerPVCNewer->>GeneratedPVCs: Apply overlay to relay PVC
DMWorkerPVCNewer->>GeneratedPVCs: Generate additional PVCs without relay overlay
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #7024 +/- ##
==========================================
+ Coverage 40.58% 41.00% +0.42%
==========================================
Files 436 439 +3
Lines 25274 25541 +267
==========================================
+ Hits 10258 10474 +216
- Misses 15016 15067 +51
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR extends the existing PVC overlay mechanism so that DM's built-in dataVolume and DMWorker's built-in relayVolume can be targeted by overlay.volumeClaims, in addition to the previously-supported additional volumes. Previously the CEL validation rejected any overlay whose name did not match an additional volume, which excluded these built-in volumes. The change loosens validation to also accept the built-in volume name and applies the overlay when constructing the built-in PVCs, while removing the consumed overlay entry before delegating to the shared additional-volume builder (which panics on unknown overlay names).
Changes:
- Broaden the CEL
XValidationrule onDMTemplateSpec/DMWorkerTemplateSpecto accept the built-in volume name (dataVolume/relayVolume) or an additional volume name, and regenerate the four affected CRDs. - Apply
OverlayPersistentVolumeClaimto the built-in data/relay PVCs, and strip the consumed built-in overlay from a deep copy before callingcoreutil.PVCs. - Add unit tests for overlay application and validation tests for the new/rejected cases.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| api/core/v1alpha1/dm_types.go | Updates CEL rule to allow overlays targeting the built-in dataVolume. |
| api/core/v1alpha1/dmworker_types.go | Updates CEL rule to allow overlays targeting the optional built-in relayVolume. |
| manifests/crd/core.pingcap.com_dms.yaml | Regenerated CRD reflecting the DM CEL rule change. |
| manifests/crd/core.pingcap.com_dmgroups.yaml | Regenerated CRD reflecting the DM CEL rule change. |
| manifests/crd/core.pingcap.com_dmworkers.yaml | Regenerated CRD reflecting the DMWorker CEL rule change. |
| manifests/crd/core.pingcap.com_dmworkergroups.yaml | Regenerated CRD reflecting the DMWorker CEL rule change. |
| pkg/controllers/dm/tasks/pvc.go | Applies overlay to the data PVC and removes it from the additional-volume overlay copy. |
| pkg/controllers/dmworker/tasks/pvc.go | Applies overlay to the relay PVC and removes it from the additional-volume overlay copy. |
| pkg/controllers/dm/tasks/pvc_test.go | Adds test verifying data + additional volume overlays are applied. |
| pkg/controllers/dmworker/tasks/pvc_test.go | Adds test verifying relay + additional volume overlays are applied. |
| tests/validation/dm_test.go | Adds validation cases for built-in data overlay acceptance/rejection. |
| tests/validation/dmworker_test.go | Adds validation cases for built-in relay overlay acceptance/rejection. |
| tests/validation/common_test.go | Adds BuiltInVolumeClaimsOverlay helper generating the shared validation cases. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
/test pull-e2e |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/validation/validation.go`:
- Around line 161-162: In the validation test before the loop over results, add
a non-empty assertion for results with the message “no CEL validation rules
found”. Keep the existing per-result require.Nil check unchanged.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 4acab17f-24f8-4341-af1e-82430b830733
📒 Files selected for processing (9)
api/core/v1alpha1/dm_types.goapi/core/v1alpha1/dmworker_types.gomanifests/crd/core.pingcap.com_dmgroups.yamlmanifests/crd/core.pingcap.com_dms.yamlmanifests/crd/core.pingcap.com_dmworkergroups.yamlmanifests/crd/core.pingcap.com_dmworkers.yamltests/validation/dm_test.gotests/validation/dmworker_test.gotests/validation/validation.go
🚧 Files skipped from review as they are similar to previous changes (6)
- api/core/v1alpha1/dm_types.go
- api/core/v1alpha1/dmworker_types.go
- manifests/crd/core.pingcap.com_dmgroups.yaml
- manifests/crd/core.pingcap.com_dms.yaml
- manifests/crd/core.pingcap.com_dmworkergroups.yaml
- manifests/crd/core.pingcap.com_dmworkers.yaml
Summary
dataVolume.relayVolume.Why
ResourceTagger derives EBS cloud tags from
tags.tidbcloud.com/*labels on PVCs. O11Y-2632914 identified three DM PVCs without those source labels because Cluster Service could not express PVC overlays for the built-in DM data and relay volumes.The first E2E runs for this PR failed before executing any specs because Kubernetes rejected the generated DMGroup CRD: the new CEL validation rule exceeded the static cost budget. This update keeps the same validation semantics with a lower-cost expression and adds regression coverage for CRD CEL costs.
How
OverlayPersistentVolumeClaimto the built-in master data PVC and worker relay PVC.Testing
go test ./pkg/controllers/dm/tasks ./pkg/controllers/dmworker/taskscd tests/validation && go test ./...make lintmake crdmake verify/crdwas attempted, but its temporary worktree failed before generation because Go resolved the macOS/varpath as/private/varand treated the build path as outside the module. The equivalent clean-worktree regeneration check passed.Risks and Reviewer Focus
Summary by CodeRabbit
Bug Fixes
Tests