KVM: Create snapshot from VM snapshot is not working on 4.17 RC3

[vladimirpetrov]

ISSUE TYPE

• Bug Report

COMPONENT NAME

API

CLOUDSTACK VERSION

4.17 RC3

CONFIGURATION

N/A

OS / ENVIRONMENT

KVM hypervisor

SUMMARY

Create snapshot from VM snapshot is not working for KVM hypervisor. It is working properly on 4.16.1.

STEPS TO REPRODUCE

Go to VM snapshots and try to create a snapshot from a VM snapshot.

EXPECTED RESULTS

If the global variable **kvm.snapshot.enabled** is set to false and the VM is running, there must be a warning message. If the VM is stopped, a snapshot must be created successfully.

ACTUAL RESULTS

Regardless of the VM state, the operation fails with the following errors:

-----
Failed to create Snapshot from VM Snapshot.
Failed to create snapshot due to an internal error creating snapshot from vm snapshot 1


022-05-27 13:29:11,427 DEBUG [o.a.c.s.s.SnapshotServiceImpl] (API-Job-Executor-42:ctx-ac02cb67 job-62 ctx-69a00fc8) (logid:8fd7fd84) Failed to copy snapshot
java.lang.RuntimeException: InvocationTargetException when invoking RPC callback for command: copySnapshotAsyncCallback
        at org.apache.cloudstack.framework.async.AsyncCallbackDispatcher.dispatch(AsyncCallbackDispatcher.java:154)
        at org.apache.cloudstack.framework.async.InplaceAsyncCallbackDriver.performCompletionCallback(InplaceAsyncCallbackDriver.java:25)
        at org.apache.cloudstack.framework.async.AsyncCallbackDispatcher.complete(AsyncCallbackDispatcher.java:126)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:534)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:84)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:106)
        at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:280)
        at org.apache.cloudstack.storage.snapshot.DefaultSnapshotStrategy.backupSnapshot(DefaultSnapshotStrategy.java:177)
        at com.cloud.storage.snapshot.SnapshotManagerImpl.backupSnapshotFromVmSnapshot(SnapshotManagerImpl.java:498)
...
2022-05-27 13:29:11,433 DEBUG [c.c.s.s.SnapshotManagerImpl] (API-Job-Executor-42:ctx-ac02cb67 job-62 ctx-69a00fc8) (logid:8fd7fd84) Failed to backup snapshot from vm snapshot
com.cloud.utils.exception.CloudRuntimeException: Failed to copy snapshot:java.lang.RuntimeException: InvocationTargetException when invoking RPC callback for command: copySnapshotAsyncCallback
        at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:301)
...
2022-05-27 13:29:11,440 DEBUG [o.a.c.a.c.u.s.CreateSnapshotFromVMSnapshotCmd] (API-Job-Executor-42:ctx-ac02cb67 job-62 ctx-69a00fc8) (logid:8fd7fd84) Failed to create snapshot
com.cloud.utils.exception.CloudRuntimeException: Failed to backup snapshot from vm snapshot
------

Further analysis shows that the failure actually comes from the host:

2022-05-30 11:41:49,506 DEBUG [c.c.a.t.Request] (AgentManager-Handler-6:null) (logid:) Seq 2-491455309336806422: Processing:  { Ans: , MgmtId: 32989492806219, via: 2, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":"false","details":"***Failed to backup snapshot i-2-5-VM_VS_20220530114135, undefined type i-2-5-VM_VS_20220530114135","wait":"0","bypassHostMaintenance":"false"}}]

***Failed to backup snapshot i-2-5-VM_VS_20220530114135, undefined type i-2-5-VM_VS_20220530114135

and the reason seems to be the **managesnapshot.sh** script:

2022-05-30 11:55:19,152 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-2:null) (logid:565d35b1) Executing: /usr/share/cloudstack-common/scripts/storage/qcow2/managesnapshot.sh -b i-2-5-VM_VS_20220530114135 -n i-2-5-VM_VS_20220530114135 -p /mnt/3ac1f06a-3f50-3e03-b53e-92faead36a85/snapshots/2/5 -t 248e458e-f2c5-48c3-86bc-5201104a5832 
2022-05-30 11:55:19,155 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-2:null) (logid:565d35b1) Executing while with timeout : 21600000
2022-05-30 11:55:19,167 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-2:null) (logid:565d35b1) Exit value is 3
2022-05-30 11:55:19,168 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-2:null) (logid:565d35b1) ***Failed to backup snapshot i-2-5-VM_VS_20220530114135, undefined type i-2-5-VM_VS_20220530114135
2022-05-30 11:55:19,168 DEBUG [kvm.storage.KVMStorageProcessor] (agentRequest-Handler-2:null) (logid:565d35b1) Failed to backup snaptshot: ***Failed to backup snapshot i-2-5-VM_VS_20220530114135, undefined type i-2-5-VM_VS_20220530114135