Poor SAN performance on RP7410 +RP7420 / EMC CX500

[Liam Curtis]

Hello all,

Trying to troubleshoot why I am getting such low IOPS (Runnning HP-UX 11.11 on RP7410 and RP7420) to some luns on an EMC CX500. Dont think it is the SAN since I have some vmware hosts which scream on the same exact luns.

I've played with qdepth and have tried using Metaluns with more spindles, and I cant seem to get good results.

Here is a sar -d 5 5

Average c0t6d0 3.08 0.50 5 99 4.47 10.55
Average c3t6d0 1.68 0.50 4 92 4.57 7.71
Average c9t1d0 66.64 0.50 142 3604 4.96 7.83
Average c11t1d1 89.88 0.50 263 6763 5.01 8.18
Average c9t0d6 0.64 0.50 2 40 4.52 6.81
Average c11t1d2 0.68 0.50 2 77 4.77 5.68
Average c11t3d5 0.04 0.50 0 1 4.60 5.01
Average c9t3d0 0.04 0.50 0 1 8.30 4.13
Average c11t2d2 0.04 0.50 0 0 0.34 14.56

c9t1d0 and c11t1d1 are the luns in question. If I am reading this correctly, that means I am only getting 6763 * 512KB = 3.3 MB/s?

Granted, it is an Oracle DB, but I am spread across 12 spindles in RAID1/0...

Using Tachyon XL2 hbas and pvlinks...no powerpath. Have tried using smaller luns with LVM striping..no better.

I am on a dirt old patchset from June 2004, but I am scared at this point to patch too much as the hardware is getting a bit older at this point and on 3rd party support.

However, if anyone has seen known issues with these HBA's and patches fixing them, I'll definitely fire up the ignite and go for it....


thnx much!

[Steven E. Protter]

Shalom,

You may need patches and drivers included in more current patch sets.

Patching your system should be the first thing you do before worrying about performance issues.

Patches often make those situations going away.

You don't mention the OS, but plan and execute installation of the OS/Gold or QPK.

Eliminate possible causes. This is one of 100 good reasons to stay current on patching.

SEP

[Jean-Luc Oudart]

Hi

I suppose you answer half the question as your VMWARE is working fine (did you measure your throuput to compare?).
Is this something new ? do you have a baseline with your Oracle IOs ?

Check this previous forum thread :

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=964202

Regards
Jean-Luc

[Jean-Luc Oudart]

Hi again

also found this hp doc
http://docs.hp.com/en/J2635-90028/J2635-90028.pdf

Regards
Jean-Luc

[Duncan Edmonstone]

I would be checking EMC's interop guide for HPUX - if you don't have the correct failover mode settings on the LUNs presented to the HPUX system using PVlinks and the correct auto-trespass setting, then the clariion can be very slow...

Can't remember specifics, but my first call would be to EMC...

HTH

Duncan

[Liam Curtis]

Thanks everyone! As this is a production server, downtime is not an option, so any patching would have to wait.

However, I did try changing scsi_max_qdepth to 8 and I am now getting decent throughput. It had been set at 4, which did not seem enough. I had it set at 32 per some recommendations on EMC forums and that seemed to clear up wio%, but it was probably throwing too much at the SAN.

GOing to check with end users today.