HP StoreVirtual VSA extremely high latency on one storage

[Tom Lyczko]

We have a two-node HPE StoreVirtual VSA setup, basically just two LUNs.

One LUN has really high latency, the other one does not, the two servers are alike in terms of configuration.

How to diagnose and fix why the latency problem??

If I think I need HPE support to fix this where/how do I find out how much this costs??????????????

We don't have any consultants in our area fluent with this product, are there any in Albany, NY????????

Or even better yet, Burlington, VT???????? Would also consider Central NY consultants.

Thank you, Tom

[oikjn]

a problem with the latency of one LUN?  Sounds like a configuration problem on that lun or its got some funky load.  The the two LUNs with the same format and the same/similar workload, or are they really different, like one is for a file server and the other is for an SQL transactional DB?

If I had to guess, the only reason one LUN would have a higher latency than a 2nd LUN on the same cluster would be because that "bad" LUN was getting forced to do excessive IO in the background because of the format structure so which is causing significant IO amplification.

[Tom Lyczko]

To the best of my knowledge the two VSAs and LUNs are set up the same way.

I moved less important VMs into the LUN with the more latency.

That's why I asked about HP support and consultants wrt this software, this is something not easily diagnosed and might be remedied by a new re-setup and so forth after reviewing what's on the LUNs etc. and how everything is set up etc.

Thank you, Tom

[Tom Lyczko]

Update: I opened a case with tech support. It was never figured out exactly why the latency issues occurred but updating some VMware settings and upgrading some firmware to more current versions was very helpful, plus setting the read-write ratio to 50-50 was helpful.

[Steven Clementi]

Tom:

What is running on the volumes? 

[Tom Lyczko]

I'm not sure what you mean., but the first answer that comes to mind is vSphere virtual servers involved with the two host servers nodes.

I don't know why the latency happened in the first place, only that it existed and that all the different things done appeared to all combine to fix the problem. It was definitely not anything to do with IOPs.

Any ideas or suggestions are welcomed for current and future references.

Thank you, Tom

[Steven Clementi]

That's what I meant..  More precisely though, how many virtual machines are on each of the two volumes... and what types of workloads are they?

Without knowing the full details of the scenario, I can not say for sure... but it could have been a pathing issue, or related to an iops setting, or an iSCSI initiator issue.

Glad things are working better though.

Are you using the StoreVirtual Multipath Extention Module?

[Tom Lyczko]

There's some imbalance of resource usage between the two hosts but it can't be helped.

I don't know anything about the 'StoreVirtual Multipath Extention Module.'

What does it do, how would it help?? Is it difficult to install?? Only two hosts so it's difficult to install things that require reboots.

The hosts are vSphere 5.5u3, another hurdle to deal with, I will upgrade the hosts before doing anything with storage.

Thank you, Tom

[ThanhDatVo]

Hello Tom,

Make sure the LUN's are configured in the same network RAID (it usually should be on Network RAID 10).

Also check the AO settings. Perhaps one LUN has AO activated while the other does not....?

Kind regards,

Thanh

[Steven Clementi]

The StoreVirtual MEM installs the appropriate settings for Multipathing in a StoreVirtual environment (The pathing used to access storage from a StoreVirtual system either physical or VSA).

Not difficult to install, but does require a reboot.  Offers better performance settings than just the native entries in ESXi, though in a small environment... you are likely not going to see a great improvement in performance.