High OVC CPU usage when performing storage vMotion

[guan8]

Hello,

We have since a month ago deployed a 2+1 simplivity federation, and have moved in our production environment into it.

Today I performed a storage vMotion on a machine + change compute resource from host 2 to host 1, and the CPU usage on the OVC on host 2 reached 100% immidiately and stayed there for 5 minutes, until I canceled the vMotion because I was afraid it was affecting our production environment. CPU usage on host 1 was much more modest, reaching a peak of 64% utilized and quickly dropping again.

Is this normal behaviour?

Best regards,

Gustav Andersson

[DaveOb]

It would be best to repeat the action at a time that is convinent gather any OVC logs  and open a service request.

Might be worth checking if the MTU for the storage and federation is capable of passing 9000mtu non fragmented.

From the ovc  ---- ping x.x.x.x -d -s 8972

[guan8]

Hi, yes, packets flow untruncated when pinging with large packet sizes from OVC on host 1 to OVC on host 2.

We only have Essentials Plus license. Could this have anything to do with it? I don't think storage vMotion is included in that. But I am confused, because we get 3 options when migrating a machine: Change compute, Change storage and Change both. When a machine is online, we cannot choose Change Storage, but if we choose Change both, it is suddenly fine and we can change storage when VM is online. This is what we were doing when the OVC displayed abnormal cpu consumption.

I'm going to try and do a live storage vMotion again this weekend when traffic is low, just to see what's going on.

[guan8]

Same behaviour experienced again when trying to change both storage and compute on a live VM.

When I try to change storage only when VM is online, I get a warning that my license (Essentials Plus) does not support that. However, when I change both storage and compute (still live), I am allowed, but the OVC flips out.

Am I missing something here? Does my license actually support live storage vMotion when changing compute at the same time? I guess that's more of a VMWare question...

Normally, a storage vMotion (even live ones?) should execute almost immidiately, since the storage in a Simplivity cluster is synchronously replicated, right?

/Gustav

[Stor_Mort]

Simplivity datastores don't work like SAN or JBOD volumes because of the dedupe and compression features. All the VMs are kept in the same pool which is synchronously replicated. A storage vMotion just copies your VM from one place to another on the same raid sets, basically changing the name of the datastore that contains the VM without changing the storage location for all that effort. Give us a little more background, there may be a better way to do what you want to do.

[tonymcmillan]

We recently deployed a pair of beefy (Dual 16-core 2.6GHz, 768GB RAM, 14.4TB) nodes and were exercising the system and had the same experience with a storage vMotion operation.

We noticed the same high CPU comsumption you did. Very high, if not 100% CPU usage for the OVCs, which had six cores each. We  were surprised. However, we were not concerned about impacting the systems as this CPU utilization is limited to only six cores out of 32.

When you tried to perform a storage vMotion and were unable, that was due to your vSphere license not supporting that feature. However, you found the same workaround we did years ago; you can do a storage vMotion as long as you migrate the host as well.

We concluded that although we thought the overhead of the OVC was very high during a storage vMotion, we concluded that it would be a rare occasion to perform this action between two SimpliVity nodes. You are correct that the nodes are already in sync so, a storage vMotion is really unnecessary. If you want to migrate a VM, you need only change the compute node. Although the data lives on both nodes, I don't think the storage Vmotion process is aware of this fact.

It sounds like you were performing a storage vMotion between different datastores, all of which are syncronized by SimpliVity. If this is the case, there is a SimpliVity Move operation that is storage aware. The only downside is the VM must be powered off to use this operation.

Thanks for taking the time to post your experiences here. We have just started our journey with SimpliVity and hoping to learn as much as possible.

Tony

[SimonHorwood]

We have an 8 node cluster and I often see high CPU warnings on an OVC when DRS kicks in a redistributes a workload. The warning doesn't persist for long