StoreVirtual Storage
1748232 Members
3305 Online
108759 Solutions
New Discussion

Performance issues when using VSA on ESX with VMXNET3 driver

 
virtualmatrix
Advisor

Re: Performance issues when using VSA on ESX with VMXNET3 driver

 

> can you suggest  > 2 TB local RDMs with ESXi 5.0?

 

I can say that it worked for some basic tests:  The VSA starts up, reports the correct amount of space, and we can create and use volumes.  I wouldn't say I could "suggest" it since we aren't using that configuration in a heavy workload/production environment yet.  We are building out another VSA based cluster and we'll need to make that decision soon.

 

>  do you also recognized the (300+) latenzys with vm version 7 ?

 

We have not upgraded any of our VSAs to version 7.  We *did* see latency symptoms in version 4, but in our case, these were resolved by using RDMs.  I suspect the original poster for this thread is experiencing a different root issue than we've seen, but it is interesting that the symptoms are similar.

 

> did you played arround with the underlaying raid level (raid 5) block size? or did you used always the default blocksize 256k? thanks for your feedback 

 

We left the controllers at their default configuration except for enabling caches.  We didn't see much benefit to increasing the blocksize given the overall random nature of I/O in a busy virtualized environment, but we did not spend time benchmarking to prove our thoughts right or wrong.  It would be different if we could better predict the most common I/O patterns for a particular group of disks.  

 

Sorry I'm not more help for those questions. 

 

If others have a different perspective on how to best optimize controllers for a VSA environment, please share!

 

 

 

M.Braak
Frequent Advisor

Re: Performance issues when using VSA on ESX with VMXNET3 driver

I got an update from vmware. They have a reproduction enviroment up and running with the vsa. Vmware managed to simulate the problem and the engineers are creating a bug report for the development team. I hope they will release a fix soon. I'll keep you updated!
5y53ng
Regular Advisor

Re: Performance issues when using VSA on ESX with VMXNET3 driver

Great news, thanks for the update Braak.

Northwoods
Frequent Advisor

Re: Performance issues when using VSA on ESX with VMXNET3 driver

As I'm currently running on some older DL380 G5's this isn't an option for me but what about using direct-path to pass the NICs directly to the VSA bypassing the problem all together?

Etavis-GNS-AG
Occasional Visitor

Re: Performance issues when using VSA on ESX with VMXNET3 driver

Having exactly the same issue as the thread starter. Any news on this?

 

Im using VSA on ESXi5 in a c7000 with ds2200 storage blades and Flex-10. Usualy perfomance is great, but when removing snapshots latency peaks to over 700ms, VMs even disconnect RDP sessions.

 

How did you change the driver to E1000? Did you change just the driver or the virtual nic of the VSA VM? Is there another way to use 10Gbit Ethernet than VMXNET3? I don't think so...

RonsDavis
Frequent Advisor

Re: Performance issues when using VSA on ESX with VMXNET3 driver

It doesn't work with all controllers. But you have to use vmfstools to get it to work if it does.

You should be able to google it. It is definitely not supported by vmware.

Tedh256
Frequent Advisor

Re: Performance issues when using VSA on ESX with VMXNET3 driver

 - do you have a vmware incident and/or bug number, that others could refer to if they experience the same issue?

 

I was hoping that if a fix was forthcoming, it would be included in the U1 which just came out last week ...

 

 

M.Braak
Frequent Advisor

Re: Performance issues when using VSA on ESX with VMXNET3 driver

My case number at vmware is 11128945812
I have asked vmware for an update. Waiting for their reply.
M.Braak
Frequent Advisor

Re: Performance issues when using VSA on ESX with VMXNET3 driver

Also refer to this thread when you create a case!!
Tedh256
Frequent Advisor

Re: Performance issues when using VSA on ESX with VMXNET3 driver

Vmware recommended using local RDM? What was the performance issue - related to vmxnet 3 nics described in the rest of this thread, and RDM route was seen as a way of by-passing whatever the issue was?

 

Huh - so I have not yet had a chance to look over the links you provided, but I'm hoping that one of them contains a list of raid controllers that export GUIDs?

 

Anyway - I generally HATE RDMs - management of them once you get more than a few is such a pain, and all the perf docs that I have seen show very little -to-none advantage of an RDM over a virtual disk file on VMDK ...

 

But in the case of a VSA, it seems to me a bit more sensible - you want the VSA to have as direct and unmediated access to the disk as possible, I would think.

 

Nice post, worth thinking about.