Adopt Detached VMware Root VMDKs as FCD Volumes - #349
Conversation
b4b7a8f to
9e40b70
Compare
|
KVM based volumes are owned by the Netapp driver, not the vmware fcd driver. |
| with excutils.save_and_reraise_exception(): | ||
| reason = _("Volume driver %s get exception.") % driver_name | ||
| status = ('error' if volume.status == 'error' | ||
| else 'error_managing') |
There was a problem hiding this comment.
why are we changing the logic here to replace 'error_managing' to 'error' here, when the failure should be error_managing ?
There was a problem hiding this comment.
This only introduces error if the driver deliberately set it before raising. The default remains error_managing.
This was an attempt to have better error-handling options in the calling code: Nova first asks the VMware compute to power off the VM and detach the root VMDK, then asks Cinder to adopt this disk via manage_existing. If Cinder fails before the VMDK backing has been relocated, Nova can still recover the source VM by reattaching the original VMDK path. The FCD driver marks those failures as error to signal that safe recovery path. Once relocation has happened, the driver leaves the error status alone and the flow results in error_managing (since at this point, manual/operator recovery may be needed and Nova can not reliably recover on its own).
Without this, Nova cannot distinguish "recover by reattaching original source path" from "Ccinder may have performed some invasive action on the disk already".
|
|
||
| def revert(self, context, result, flow_failures, volume, **kwargs): | ||
| reason = _('Volume manage failed.') | ||
| status = 'error' if volume.status == 'error' else 'error_managing' |
|
Also the comment in this PR says "We want to move image-backed OpenStack servers from VMware to KVM. Those servers boot from Nova-owned VMDKs on VMware ephemeral datastores. KVM cannot use those disks directly. Before the resize can continue on KVM, the root VMDK needs to become a Cinder-managed disk on an NFS-backed datastore. " Cinder doesn't have access to non-kvm ephemeral datastores. No driver manages those, even though they are in vcenter. |
Fully agreed that normal KVM-side customer volumes are expected to be owned by the NetApp driver. This PR provides an intermediate step between VMware and KVM: The goal is to get that disk into Cinder management in a format that can be attached to both hypervisor types. The nova-managed disk is registered/imported as an FCD and moved onto an NFS-backed FCD datastore, so the VMware instance can become BFV before the migration to KVM continues. By taking advantage of #288, we can also attach that FCD volume to both hypervisors until the migration is complete. At that point, the disk should be properly retyped to be under control of the NetApp driver. I will clarify the PR description to avoid implying that this creates the final KVM/NetApp-owned volume. |
The code uses the vCenter session to resolve the source datastore by name and pass the detached VMDK path to vSphere I tested this in QA with a root VMDK on If this is about the PR body statements rather than the mechanics, I'm happy to adjust that to avoid ambiguity! |
Let drivers signal safe pre-relocate failures by setting volume status to 'error' so the manage_existing flow preserves that state on rollback instead of always reverting to 'error_managing'. Add recovery logging in ManageExistingTask when the DB update cannot be persisted after a driver has already adopted the backend object. Change-Id: I4d060160561fd8df4e1869194a14934edf40f060
Add manage_existing and manage_existing_get_size overrides to VMwareVStorageObjectDriver. Parses the '[ds] path.vmdk' source-name ref, validates caller-provided size_gb, registers the VMDK as a First Class Disk via RegisterDisk, and relocates it to the target datastore when source and target moref differ. All pre-relocate failures mark the volume as 'error' (safe for Nova abort); relocate and post-relocate failures leave status at the flow-owned 'error_managing'. No error path calls delete_fcd or unregister_disk. Change-Id: Ic7faba0b8aefaff1472a5ff0d380547c856f1a63
Problem: The volume_extension:volume_manage policy defaults to rule:admin_api, so only admin users can call the manage_existing API. When another service (e.g. Nova) calls manage_existing on behalf of a user, the request carries the user's token as primary (for correct volume ownership and quota attribution) plus the service's token as secondary. The non-admin user token is rejected by the policy. Solution: Before enforcing the admin-only policy, check whether the request carries a valid service token using the existing is_service_request() helper (checks ctxt.service_roles against the configured service_token_roles). If the request comes from a trusted service, skip the policy check. This mirrors the pattern already used in attachment_deletion_allowed() for service-initiated detach operations. The admin path and the non-admin-without-service-token path remain unchanged. Change-Id: I88b07712f399dae7654d238605f6f410a1b5a9ba
9e40b70 to
b9305f8
Compare
|
I am planning on rolling Epoxy to qa-de-1 on Monday. Can we rework this PR on top of 2025.1-m3 ? |
|
Cherry-picked all of the commits onto |
2 participants
We want to move image-backed OpenStack servers from VMware to KVM. Those servers boot from Nova-owned VMDKs on VMware ephemeral datastores. KVM cannot use those disks directly. Before the resize can continue on KVM, the root VMDK needs to become a Cinder-managed disk on an NFS-backed datastore. We are using FCD as an intermediate representation that can be attached to both VMware and KVM hosts (via #288). This PR adds the Cinder side of that conversion.
The FCD driver now has its own
manage_existingimplementation instead of inheriting the VMDK driver's shadow-VM path. It accepts a detached VMDK reference in[datastore] path.vmdkform. If the ref carries an optionalsource-id(the vSphere FCD id, passed by Nova whenVirtualDisk.vDiskIdis present before detach), the driver validates the FCD backing path matchessource-nameand renames it; otherwise it callsRegisterDisk. In both cases the FCD is then relocated if the selected Cinder datastore differs from the source. The returnedprovider_locationpoints at the final FCD location, so the existing FCD attach paths keep working.If
RegisterDiskreports the disk is already registered and nosource-idwas provided, the driver fails clearly withManageExistingInvalidReferenceand tells the caller to passsource-id.The shared
manage_existingflow preserves the volume status set by the driver on rollback. Pre-relocate failures mark the volume aserrorso Nova can abort and reattach the original VMDK. Failures during or after relocate end up inerror_managing, where an operator needs to recover the managed FCD.The manage API policy (
volume_extension:volume_manage) accepts service-token callers via a customis_service_request:Trueoslo.policy check registered incinder/policies/base.py. This allows Nova to callmanage_existingon behalf of a user without requiring the user to have the admin role. Deployments that override this policy inpolicy.yamlmust addor is_service_request:Trueto their override.The storage-side safety rule is simple: never delete the disk while adopting it. The new path does not call
delete_fcdor unregister anything on failure. It logs source path, FCD id when known, target datastore, and failure reason.Nova will call this later from the image-backed VMware-to-KVM resize pre-step. Until then it is inert; nothing calls this FCD manage path unless a caller explicitly passes a detached VMDK reference to the manage API.