XP (12/10K, 512,1024) SAN Boot Disks on HP 11.11 Behaviour - Massive Disk Queuing but otherwise Ok?

[Alzhy]

To those that have ventured in to using SAN boot disks - particularly XP SAN arrays - have you scrutinized/check your bootdisk performance at all? I am noticing during heavy use of the OS partitions (runing LVM/PVlinks btw) - i.e. software and patch installs - "sar -d" and glance show massive disk queuing! I am getting values of 40-50!

But aside from the queueing throughput is very fast and there are no other problems. I can install/ignite my net_recovery_archive or Golden Image (average 8GB) to my SAN disks in under 20 minutes. It's just the queuing is so high but otherwise evrything is normal..

[TwoProc]

Are those lvols for each disk (or even each vg) one big LUN that you luse'd together? If so, then what you're seeing is what I'd expect to see and that's fine. Just check to see if your scsi queue depth is high enough to take it.

[Alzhy]

Messr. Joubert,

The LDEV(Lun) I am using is not a LUSE but an Open-V slice of a 300GB 6+2P aray group. It is sized 50GB of which 16GB is swap while the rest are for the OS and nothing else.

I tested with scsi qdepth set at 8, 16, 32, 64 with no difference in the queuing values either in glance or "sar -d". And no difference in the quickness of my ignite, patch application and software installs.

SAN boot ignitions (8GB Golden Image) to XP SAN LUNS (using Tachyon XL2 Cards - A6795?) averages 15-18 minutes. To A 15K SCSI (U320) SCSI disk on a DS21XX enclosure -- it is anywhere between 30 to 40 minutes.

But the real advantage is we can "easily" relocate environments to any compatible server or frame. I am as a matter of fact re-writing/re-archiecting my client's on-site DR as well as off-site DR (with a posible Dark Fibre on the horizon)...

Stay tuned.

I am however still puzzled by the queuing..

[RAC_1]

The boot disk from SAN, is it spreat over different controllers?? Is it a single disk on SAN or comprises more than one disk on SAN?? What does SAN data say?? Glance or sar may see one SAN disk with high queue, but this one disk may be spread over multiple disks on SAN. sar ot glance has no way to look at it.

[TwoProc]

Well, I always cut my volumes into smaller chunks (I'm probably one of the few fans of the "many" lun theory), and present them all to the system. Then, I use distributed allocation to stripe amongst all of the disks that I would have presented as a single lun to the disk. And, the result is - I never see large queue depths like that. But, I have lots o' luns so I would expect to see short queue depths for each lun if I've managed to distribute correctly. So, I think what your seeing is normal, and I would recommend leaving the scsi queue depth at a level of at least what your average queue depth is, and maybe a bit more. You might not be able to measure a difference (though I'd think you could), but it would be nice to know that you did all that you could to avoid latency as possible. However, it is for the very above reason that you stated that HP technicals (many of them) recommended the strategy that I indicated above. Their contention was that having a relatively high number of separate scsi queues made for a better performing system than the latter. I did not however, benchmark that configuration difference (though I wish I had so that I could definitivly say one way other the other) - HP claimed to have done so in their labs on Oracle databases in a variety of scenarios.

[Alzhy]

Still having huge queue values when doing mostly writes to the OS partitions but the system is just rock solid and fast! Software installs are very fast compared to their SCSI counterparts.!!

My XP BootLUN presented over dual 2G HBAs is sized 50GB. Using LVM and 16GB is allocated as primary swap.