Secure path / disk queue issue

[TTaylor]

Hello -

I have an issue where on heavily utilized MS Exchange servers intermittently the disk queue length will get high (100+, 48 spindles). Doing a failover of secure path will resolve the issue almost instantly.

Our storage team indicates that they don't see any queue issues on the SAN and fibre network is not being overloaded.

The HBAs are Emulex in Proliant DL580 G3 servers. I believe the SAN is an EVA5000.

Any suggestions, or experience is much appreciated.

-T

[Nigel Poulton]

Hi,

I would recommend a good place to start would be your HBA and its driver and firmware versions etc. Are they on the support matrix for your SAN and EVA firmware configuration? Are the drivers old? Are you using scsiport or storport miniport driver? Get your storage team to help you with making sure everything is supported.

If your storage guys are telling you they dont see any issues on the SAN and storage then the I/O could be queueing on the HBA.

I would also suggest that you look and see if you get the queued I/O on the same path all the time or if it is randomly on both all paths. If it is always on a the same path then you have starting point.

Hope this helps

[Craig Howe_1]

As already mentioned the driver set and firmware is vital. What version of Secure Path do you have (latest?)

Based on you have an EVA 5000, what version of firmware on the controllers? 3.025/3.028? Ver 4.x I believe bought about a bunch of back ported 'next gen' type features to the EVA 3000/5000 such as active-active controllers so the HBAs can load balance.

Ver 4 is a decent upgrade and there is documentation from Hp to check all the various aspects of the upgrade.

Back to the HBAs - If you have a small number of hosts on the SAN, you could look at pushing the queue depth on the HBA from anywhere from the default (16?) to 255. This can be done through the Lightpulse utility lputil and can speed up things considerably.

When you mention about the metrics you have used, Im not sure if this is through Windows Perf monitor. Are you guys also running EVAperf? You can use this to monitor the SAN and hosts to pickup any performance problems. Things like host port queue depth, controller port depth, cache, virtual disks, write and read latency from the SAN can be measured. There are also Mibs for the switches that you can plot in monitoring apps like RTG/Cacti/MRTG, etc. This along with EVAPerf will help pinpoint if there are any bottlenecks in either the host, fabric (not usually) or the storage system

[Nigel Poulton]

Hi TTaylor,

Did you ever get this resolved? I would be interested to know what the problem and solution was.

mackem