StoreVirtual Storage
cancel
Showing results for 
Search instead for 
Did you mean: 

SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

 
ThomasU2
Advisor

SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

Dear all

Creating a snapshot (or deleteing one) needs a very, very long time until finished. Sometimes a turned off VM needs 90 seconds to create snapshot! mostly about 50seconds for a turned of VM.
also a turned on one even longer (without RAM or quiesce).

deleting a snapshot needs 10times more than other attached iscsi storages (different models)

Has aynone the same problem? If yes, any solution?

Esxi6, 3 servers, cluster, sv3200 with updates installed. LUN is 2.5TB and 10GB iscsi connected.

10 REPLIES
oikjn
Honored Contributor

Re: SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

that isn't normal.  can you provide any additional information about your setup?  Are there any errors reported?  how many snapshots do you have outstanding?  How many actions are happening at once?

 

Watch the node CPU level, snapshots are CPU intensive so you should see that activitiy spike during the creation, but if its spiked before that could be a sign of something else going on too.

ThomasU2
Advisor

Re: SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

Hi Oikjn

SV3200 with 10GB network attached to 2x1950 HPE switches.

3xHP dl360 G8 servers with ESXi 6 on it.

1 LUN with 3TB. At the moment 1 VM on it, turned off, 50GB size.

No other traffic to/from the sv3200, still not in productive enviroment.

Open case with HPE, network team said, our network is ok, theres no issue.
HPE Storageteam has NO CLUE and seems they do not care. I bought this 6 months ago, opened a case 4 months ago. And it's still not working as expected. at the moment it needs 2 minutes to create a VM snapshot from ESXi from a TURNED off VM.  (yes, 2 minutes, from a turned off VM, something I've never seen in 10 years of handling virtual enviroments)

I just hoped somebody has the same problem here with a Storevirtual 3200....

 

 

oikjn
Honored Contributor

Re: SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

how are you crating the snapshot?  Is this application created or application aware?  Sounds like a timeout communication issue.  What if you create a manual snapshot that is not application aware through CMC?  Snapshot creation ususally takes seconds at most.

 

I"m an all M$ shop, so I can't help much on the troubleshooting side for this one, but I'd be looking closely at the logs in the hypervisor to see if anything there is getting stuck or what the last logged step is when a snapshot is requested if you are doing it through esx.

Re: SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

are you using VEEAM in your setup? 

is there any security scan software scanning SV3200 host ports? can you try disable of that scan and then try to take snapshot to see how it progress

how are you creating snapshot? need more detail

Is it possible for you to share SV3200 Support bundle or share HPE Case id so that I can access the log?

 

Hope this helps!
Regards
Subhajit

If you feel this was helpful please click the KUDOS! thumb below!

***********************************************************************************

ThomasU2
Advisor

Re: SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

No veeam invovled

No security scan involved.

Hmm..I open vmware vcenter, right click on the Virtual machine - Take snapshot - Nameit - click OK.

This Case is ongoign for about 4 months. We have done a lot of things, and updates etc, etc. Are you really prepared to read through the whole case? ;-) If yes I can give you the caseID if you want.
As I said, I just wanted to know if somebody with a similar configuration stumbled on the same problems

Esxi cluster with 1GB copper to a switch, SV3200 with 10GB fibre to switch. and vmware vcenter snapshots taken 90seconds to proceed on a turned off VM. Any other storage (and I have 3 others, iscsi and NFS) need about 1-2 seconds.

oikjn
Honored Contributor

Re: SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

Try using CMC to create the snapshot and not vcenter and do NOT select application aware and see how long that takes to create.  My guess is that will be fast and the issue is likely some communication issue between vcenter and the management group.

ThomasU2
Advisor

Re: SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

Hi oikjn

That is NOT the problem I describing. There is no CMC and those are not lefthands. Its Storevirtual 3200 (not 3000 or any other number). I don't want to snapshot a LUN. I want to Snapshot a single virtual machine with ESXi vcenter.

regards
Thomas

 

oikjn
Honored Contributor

Re: SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

interesting.  not sure why one random storevirtual device has a different management method than the others.  I don't know the SV3200, but I would have assumed it was still iSCSI.  I'm also only familiar with hyper-v shapshots, but I"ve never heard of VM level snapshots done by the SAN.  

I'm a Hyper-V shop so excuse my ignorance, but at least for MS, the creation of a VM snapshot simply generates a .avhdx file where changes are written instead of to the .vhdx file.  As far as the SAN is concerned, this is just normal IO traffic.  The creation of those files can be IO intensive, but is not something that the SAN is specifically aware of or manages, so unless you are seeing high latency issues during that VM snapshot creation time, there is not really a "SAN issue".  

At least for the other SV units and any other SAN i've heard of, Snapshots on the SAN must be done at the LUN level.  When I see our backups create hardware assisted snapshots for backups of VMs, I see the entire LUN get the snaphot and then it is released after the backup of that VM(s) is completed.

 

IF the SV is truly different then ok forget everything above, but even if your goal is to snap the individual VMs, you should still TEST the generic snapshot functionality and manually create a snapshot using your SAN management interface you do have.

ThomasU2
Advisor

Re: SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

Hi oikjin

I appreciate your answer but as I said. It's not the problem. I am able to snaphshot a VOLUME on the Webinterface of that SV3200, a VOLUME. It's quite fast and ok. But NOT my problem.

I am fully aware what you want to tell me. but as I said, not the problem. It's very specific.

We have a 3TB Volume on that sv3200. It's attached via iscsi to the esxi cluster. The backupsolution is using VM Snapshots, the same way as when you create a manually one, in VMWare VCenter.

And yes, it's the same as hyper-v, it just creates an additional file or two on the storage. We don't know why it is so slow. I have a 5 year old Synology with 3 sata HD's, and a snapshot of a turned off VM needs 2-3 seconds...on the SV3200, it needs around 60 seconds. If I just copy files from/to the sv3200 its speed is...let's say acceptable...

So you see, it's only when creating a snapshot within VMWare vcenter. And my "only question" was, if someone with the same configuration (esxi cluster, sv3200 with iscsi attached) has the same "slowliness", nothing more. Because if others have that too, the HPE support may be take me a little bit more serious. Because that slow snapshot thinggy leads to a timeout in backing up the VM's and the Supportcase is now open for OVER 3 months...

regards
Thomas

oikjn
Honored Contributor

Re: SV3200 and Esxi 6 cluster. Snapshot VM needs very long (delete as well)

Thomas,   Is vmware creating a snapshot on the iSCSI lun when it creates its snapshot or is this purely a file created on the iSCSI LUN?  If no snapshot is created on the SAN, then this is either a SAN performance issue or a VMWare related issue.  If no SAN snapshot is created during the VM snapshot, the SAN is getting "normal" IO traffic, and unless you are seeing a sudden spike in outstanding IO and latency or some other indicator that the SAN is suffering from some under-performance issue during your snapshot creation activities, then the issue is more likely with VMWare and how it is handling the creation of those snapshot files.

If you haven't already and the HP SAN performance monitor doesn't indicate any obvious performance issues, I'd suggest opening a case with vmware about the issue as they should be able to say why the snapshot creation is taking so long to create.