AIX 5.2 eva8000 scsi queue depth

[Peacebwitchu]

I know there were some issues with the scsi queue
and previous versions on the eva firmware. We have upgraded to XCS 6.x and ODM MPIO fileset 1.0.1.0 and set the queue depth to 255. We have seen little if any improvement from going from a queue depth of 8 to 255 we did see an improvment going from a queue depth of 1 to 8. Can we set a queue depth higher than 8 and will the eva accept it? Also are there any other tweeks to AIX to perform for maximum performance on the EVA?

[Amar_Joshi]

Queue depth on host is independent from the storage queue depth. Queue length on host HBA is the IOs kept in waiting state to be transmitted. Queue length on storage is what is being kept in waiting to be written onto the disks.

On Windows servers, a default queue depth is 32 and can be increased to 128 or even 255. If the application developers have any recommendation then set it accordingly on the HBAs but if not, you may wanna try increasing it to the next step and try it out.

[Tom O'Toole]

The EVA can definitely do queue depths higher than 8, but if your system load is such that you don't consistently have 8 or more outstanding IOs to the device then you are not using the larger queue.

Is IO performance poor, what throughput are you getting, and is that considered poor for your eva configuration? How many physical disks are available for your LUNs, how many fcHBAs, etc....

Have you checked the traffic at the switch on each aix host port and eva front end ports for balance? With the HP MPIO for EVA, you will only use one path at a time per LUN. You can load balance multiple LUNS, however

[Peacebwitchu]

We have 1 terabyte lun on this system and we are currently getting 120 MB's/sec peak. I think this is pretty good I'm more curious about wether I can get more than a scsi queue depth of 8 on AIX with then new ODM fileset. This is not really documented nor was the fact that you couldn't have a scsi queue depth of more than 1 with the older firmware on AIX.

[Tom O'Toole]

I think it can go up to 255, whether that's honored by all layers in the system is a good question. But because only one path is used for your single large LUN, you could possibly see better performance with several smaller luns providing the storage for a single large lv on aix.

[Bret Graham · Accepted solution]

With your XCS version and the MPIO 1.0.1.0 you can certainly set the queue depth higher than 8. The highest value tested was 64. 8 was chosen as the default as it would apply to most situations.

I would also agree with Tom that a single 1TB LUN is maybe not the best setup. I could easily see a lot of contention for that one LUN depending on what is stored on it. I would also hope that the disk group on the EVA that holds this 1TB vdisk is comprised of lots of disks to spread the load out across lots of spindles.

Bret

[Peacebwitchu]

Thanks for the info. My next step was to test several small luns in a VG and compare performance. I am not the SAN Admin but I believe that we are using very large disks as well.

[Uwe Zessin]

Many enterprise Unix applications *love* many disks.

We've seen this recently at a customer who runs SAP on DB2 on AIX on EVA. The database *needs* many disks in order to convince it to do multithreading, I have been told. A large I/O queue will not help, because the database just does not start that many I/Os in parallel on a single disk.

If I remember correctly, he is now at about 700 virtual disks per EVA.

[Tom O'Toole]

Once you have created several hdisks you will want to balance the paths - you could still have all hdisks I/O going over a single path. The EVA mpio fileset includes an, as far as I know, undocumented utility to help with this called lbhba:

lbhba -e ALL

will write the chpath commands to stdout that can be run to set the path priority (1 is highest). The output of this utility might need to be hand edited a bit, but what you want is a different primary path with priority 1 on each different hdisk, with other paths for that hdisk set to priority 2 .

Please let us know if balancing on many luns raises your throughput at all.