SimpliVity OVC Shutdown times

[CadenLange]

Hi

My experience with OVC shutdown duration is that it takes quite a long time and I'll like to know if there are ways to speed this up.

With a two node cluster I can shutdown all the running VM's in about 5mins (I don't have that many) and this will leave me with just the 2 OVCs. I shutdown the first OVC using he svt-shutdown-safe option and this can take about 10-15mins. It seems a little quicker if I using the vCenter plugin to shut it down rather than SSH commands - is that expected?

However, the second OVC (which I must use SSH for as I have noe shutdown the vCenter server) takes about 20mins. So this means that shutting down the two OVCs can take over 30mins  - and this is after I've already stopped all my virtual machines. Is there a quicker way to gracefully shutdown the OVCs once I have no running VM's? I don't want to risk just powering off as this may impact the integrity of the environment when I start things back up again

many thanks

[gustenar]

Helllo @CadenLange 

Thank you for using HPE Simplivity Forum. 

The procedure you are using is the correct one to ensure a graceful shutdown of the OmniStack Controller. It takes time due to the internal operations it has to perform before shutting down the OVC. In general 15-20 minutes shutdown time is considered normal for the OVC.

Thanks.

[jlangmead]

Hi

Many thanks for the reply.

I just want to confirm that there isn't a way to reduce the shutdown time if it can be ensured that all the guest VMs are already shutdown - are there any manual commands perhaps to stop the svtfs and then to shutdown the OVC quicker?

The main reason for asking is that I need to reduce the UPS runtime for this procedure and I can shutdown all my vm's in under 10 mins but it then takes a further 40mins to shutdown the OVCs - seems a little crazy. I don't want to risk any corruption by risking the UPS run out of battery power before the OVCs and ESXi hosts have shutdown but equally struggling to justify the need for additional UPS runtime batteries just to allow the time for the OVCs to shutdown within an already shutdown environment.

thanks in advance

[Randolfini]

Hi Jlangmead,

I found that extended OVC shutdown times can be caused by OVC losing access to the arbiter and vCenter.

Are you trying to automate an auto-shutdown in case of power loss?

Are you running the arbiter and vCenter on this two node cluster?

If so, then here are my recommended steps to shutdown:

1. Migrate Arbiter and vCenter (either VCSA or Windows VM) to local storage of one of the two nodes.

2. Shut down the remaining VMs except the vCenter, Arbiter, and OVCs.

3. On the node NOT hosting the vCenter and Arbiter, gracefully shutdown the OVC

4. On the node HOSTING the vCenter and Arbiter, both of which are now on local storage, gracefully shutdown the OVC

5. Both OVCs should shut down in a reasonable time, I believe faster than 20 minutes each

6. Now you can shutdown gracefully the vCenter and Arbiter and proceed with host shutdowns if necessary

You don't want to force the shutdown faster with any commands. If it takes a long time then chances are the OVC(s) have split-brained because of lost access to vCenter and Arbiter.

[jlangmead]

Hi Randolfini

Yes - I need to automate the shutdown using the UPS functionality when it's running on battery and need to make sure the shutdown completes before the UPS battery is exhausted. The arbiter is running on a different server (not on the cluster), however vCenter is running on the cluster.

You're absolutely correct on shutdown times being reduced when vCenter is still up and the interesting thing is that both OVC's shutdown in under 3 mins in that scenario. The first OVC is fine as I can shut this down whilst vCenter is on the shared datastores and managed by the other node. The second OVC is still a problem though as it takes just as long to storage vmotion the vCenter server to the nodes local datastore as it does to shutdown the OVC without vCenter running - plus I don't really want to use the local datastore for anything other than the OVCs as HPE don't really like this - albeit even for a temporary location to help with the shutdown. Also it then means it needs to be moved off again when powering everything back on - but it was useful to do this to test the theory.

It does beg the question though as to what the last OVC is doing during the shutdown that means that it can shutdown in 3mins when vCenter is running but takes 20mins when it's not. Or perhaps it's to do with running the shutdown command as the svtcli user rather than administrator@vsphere.local??

thanks again - very interesting to know that actually shutdown times can be reduced to just a couple of minutes per OVC if vCenter can remain accessible to the final OVC.

thanks Jon