A6826A Fiber channel slower disk IO then A6795A

[Luk Vandenbussche]

Hi,

We have to server with the same application and same disk setup.
Server 1 : rp5470 with dual A6795A fiber controller
Server 2 : rp4440 with dual A6826A fiber HBA.
Both the servers are connected a different DS2405, the Logical volumes are the same and the speed off the DS2405 is 2 GB.

But the disk IO on the rp4440 is 3 times slower then on the rp5470.

We installed the latest driver B.11.11.05 for the card A6826A on the rp4440.

What can you do the speed up the IO on the rp4440?

[RAC_1]

Is the firmware same?? Are the disks set exactly the same way??

[Luk Vandenbussche]

I'am 100 % sure that the disk setup is the same. We double checked it with three persons.
Which firmware? How can I check it

[Zeev Schultz]

Post output of "fcmsutil /dev/tdX" (where X is the number 0,1... get from output of 'ioscan -fnCfc') for both a6795a and a6826a.

a6826a is a dual port.both ports behave the same?

It requires certain patches (see below) :
-11.11
PHKL_28984 or later (required for fcmsutil)
PHKL_28569 or later (WSIO, not required if on HWE0306 or later)
PHKL_29468 or later (required for OLA/R support)
PHKL_28695 or later (VM, recommended for better performance)

[Mahesh Kumar Malik]

Hi

1. Check switch setting on LCC Card of DS2405 connected through A6826A. Data Transmission SW should be set to 2Gb Mode.

2. If LED on A6826A is Orange, it is communicating 1Gb speed, if Green speed is 2Gb

Regards
Mahesh

[Luk Vandenbussche]

The speed is 2 GB (see above)
I am waiting for the output off the requested commands

[Zeev Schultz]

Both fcmsutil /dev/tdX (fro a6826a) and ds2405 show 2Gbit/sec (can also check DS2405 via "cstm" , "sel dev" to line where "A6250A" appears and "info" , "il") ?

Did you verify that cables are ok? Could also see from fcmsutil /dev/tdX stat - check "Link fails", "loss of signal".

[Stuart Abramson]

EMC recommends that we set a kernel parameter, which I forget right now, but it's something like "max_scsi_disk_queue_depth". Could that be it?

[Luk Vandenbussche]

Here is the requested info

[Luk Vandenbussche]

2nd file

[Zeev Schultz]

I doubt that this is the parameter name but would suggest (after verifying there is no FC link problem and that ds2405 controller is ok)
running "sar -d" on both servers and compare queues etc. Kernel parameters can be seen later with "kmtune".

[Zeev Schultz]

fcmsutil looks ok for a6826a.Link as stated is 2GB/s, private loop, Driver version higher than B.11.11.04.

I would suggest:

1) check with fcmsutil /dev/tdX stat for link errors (even compare with a6795a)
2) check performance with diskbench, link here:
http://h21007.www2.hp.com/dspp/tech/tech_TechSoftwareDetailPage_IDX/1,1703,5971,00.html

with what do you check your performance now?

3) check I/O with "sar -d", compare with rp5470.

[Nico Schockaert · Accepted solution]

Luk,

Be sure you are on HP05 for the disk-firmware. Check with diskinfo.

Regards,

Nico

[Luk Vandenbussche]

Here you find the output of

fcmsutil /dev/fcd0 / 1 stats from the rp4440

Isn't the number or Interrupts received not very high?
What can I do about i?

[Yew Lee]

Did you install the FC card on the "faster" PCI slot? The FC card should be located on the right side (at the rear). Those are dedicated 133Mhz PCI slot. The rest are shared PCI at bandwith of 66Mhz or lower.

[Nico Schockaert]

Luk,

Slot 7 and 8 are PCI-X 133MHz 64Bit.
Those are HP-UX paths 0/2/1 and 0/6/1.
As such, it seems that one of the FC cards are in a PCI-X 66MHz 64Bit. I am not sure that this is the real bottleneck, but nevertheless, it is better to put those cards in the right slots.

Regards,

Nico

[Luk Vandenbussche]

Nico,
Thx for the feedback about the slots.

BTW the server is installed by an HP Engineer ..

[Zinky]

You mentioned "the Logical volumes are the same" and obviously both are connected to similar DS2405s. But how sure are you that both are configured the same?

1. How are the logical volumes created on both servers? LVM or VxVM? Stripe or RAID5?

2. How did you test and what gave you the conclusion that the A6826A is "slower"?

3. Can you do compare:


time dd if=/dev/rdsk/cXtYdZ of=/dev/null bs=1024k

on a disk hooked up to the DS2405s on each server?

[Ted Buis]

Make sure that the kernel parameter, default_disk_ir is the same on both servers. I have seen up to 4X difference in peformance in write intensive applications. However, consider the risks of having it set to 1.

[Luk Vandenbussche]

Ted,

This parameter is not set.
If you have more ideas

[Zinky]

Luk,

Can you ask your admin (seems you're not directly admin'ing the machines) - to compare dd disk performance between the 2 machines?

Also, proof that indeed the LVM config (and filesystems) on the 2 machines are indeed set up "similarly"?

[Ted Buis]

Compare the mount options to see if they are the same in /etc/fstab. Are the same number and type of disks in each volume group?

[Luk Vandenbussche]

There was a difference in the buffer cache between the two servers.