topic Re: A6826A Fiber channel slower disk IO then A6795A in Operating System - HP-UX

A6826A Fiber channel slower disk IO then A6795A

Luk Vandenbussche — Tue, 20 Sep 2005 06:18:08 GMT

Hi,

We have to server with the same application and same disk setup.
Server 1 : rp5470 with dual A6795A fiber controller
Server 2 : rp4440 with dual A6826A fiber HBA.
Both the servers are connected a different DS2405, the Logical volumes are the same and the speed off the DS2405 is 2 GB.

But the disk IO on the rp4440 is 3 times slower then on the rp5470.

We installed the latest driver B.11.11.05 for the card A6826A on the rp4440.

What can you do the speed up the IO on the rp4440?

Re: A6826A Fiber channel slower disk IO then A6795A

RAC_1 — Tue, 20 Sep 2005 06:27:45 GMT

Is the firmware same?? Are the disks set exactly the same way??

Re: A6826A Fiber channel slower disk IO then A6795A

Luk Vandenbussche — Tue, 20 Sep 2005 06:33:17 GMT

I'am 100 % sure that the disk setup is the same. We double checked it with three persons.
Which firmware? How can I check it

Re: A6826A Fiber channel slower disk IO then A6795A

Zeev Schultz — Tue, 20 Sep 2005 06:37:28 GMT

Post output of "fcmsutil /dev/tdX" (where X is the number 0,1... get from output of 'ioscan -fnCfc') for both a6795a and a6826a.

a6826a is a dual port.both ports behave the same?

It requires certain patches (see below) :
-11.11
PHKL_28984 or later (required for fcmsutil)
PHKL_28569 or later (WSIO, not required if on HWE0306 or later)
PHKL_29468 or later (required for OLA/R support)
PHKL_28695 or later (VM, recommended for better performance)

Re: A6826A Fiber channel slower disk IO then A6795A

Mahesh Kumar Malik — Tue, 20 Sep 2005 06:45:57 GMT

Hi

1. Check switch setting on LCC Card of DS2405 connected through A6826A. Data Transmission SW should be set to 2Gb Mode.

2. If LED on A6826A is Orange, it is communicating 1Gb speed, if Green speed is 2Gb

Regards
Mahesh

Re: A6826A Fiber channel slower disk IO then A6795A

Luk Vandenbussche — Tue, 20 Sep 2005 06:48:26 GMT

The speed is 2 GB (see above)
I am waiting for the output off the requested commands

Re: A6826A Fiber channel slower disk IO then A6795A

Zeev Schultz — Tue, 20 Sep 2005 07:12:58 GMT

Both fcmsutil /dev/tdX (fro a6826a) and ds2405 show 2Gbit/sec (can also check DS2405 via "cstm" , "sel dev" to line where "A6250A" appears and "info" , "il") ?

Did you verify that cables are ok? Could also see from fcmsutil /dev/tdX stat - check "Link fails", "loss of signal".

Re: A6826A Fiber channel slower disk IO then A6795A

Stuart Abramson — Tue, 20 Sep 2005 07:17:10 GMT

EMC recommends that we set a kernel parameter, which I forget right now, but it's something like "max_scsi_disk_queue_depth". Could that be it?

Re: A6826A Fiber channel slower disk IO then A6795A

Luk Vandenbussche — Tue, 20 Sep 2005 07:40:41 GMT

Here is the requested info

Re: A6826A Fiber channel slower disk IO then A6795A

Luk Vandenbussche — Tue, 20 Sep 2005 07:41:03 GMT

2nd file

Re: A6826A Fiber channel slower disk IO then A6795A

Zeev Schultz — Tue, 20 Sep 2005 07:51:35 GMT

I doubt that this is the parameter name but would suggest (after verifying there is no FC link problem and that ds2405 controller is ok)
running "sar -d" on both servers and compare queues etc. Kernel parameters can be seen later with "kmtune".

Re: A6826A Fiber channel slower disk IO then A6795A

Zeev Schultz — Tue, 20 Sep 2005 08:33:33 GMT

fcmsutil looks ok for a6826a.Link as stated is 2GB/s, private loop, Driver version higher than B.11.11.04.

I would suggest:

1) check with fcmsutil /dev/tdX stat for link errors (even compare with a6795a)
2) check performance with diskbench, link here:
http://h21007.www2.hp.com/dspp/tech/tech_TechSoftwareDetailPage_IDX/1,1703,5971,00.html

with what do you check your performance now?

3) check I/O with "sar -d", compare with rp5470.

Re: A6826A Fiber channel slower disk IO then A6795A

Nico Schockaert — Tue, 20 Sep 2005 10:17:11 GMT

Luk,

Be sure you are on HP05 for the disk-firmware. Check with diskinfo.

Regards,

Nico

Re: A6826A Fiber channel slower disk IO then A6795A

Luk Vandenbussche — Wed, 21 Sep 2005 00:56:50 GMT

Here you find the output of

fcmsutil /dev/fcd0 / 1 stats from the rp4440

Isn't the number or Interrupts received not very high?
What can I do about i?

Re: A6826A Fiber channel slower disk IO then A6795A

Yew Lee — Wed, 21 Sep 2005 01:14:07 GMT

Did you install the FC card on the "faster" PCI slot? The FC card should be located on the right side (at the rear). Those are dedicated 133Mhz PCI slot. The rest are shared PCI at bandwith of 66Mhz or lower.

Re: A6826A Fiber channel slower disk IO then A6795A

Nico Schockaert — Wed, 21 Sep 2005 02:59:13 GMT

Luk,

Slot 7 and 8 are PCI-X 133MHz 64Bit.
Those are HP-UX paths 0/2/1 and 0/6/1.
As such, it seems that one of the FC cards are in a PCI-X 66MHz 64Bit. I am not sure that this is the real bottleneck, but nevertheless, it is better to put those cards in the right slots.

Regards,

Nico

Re: A6826A Fiber channel slower disk IO then A6795A

Luk Vandenbussche — Wed, 21 Sep 2005 08:27:25 GMT

Nico,
Thx for the feedback about the slots.

BTW the server is installed by an HP Engineer ..

Re: A6826A Fiber channel slower disk IO then A6795A

Zinky — Wed, 21 Sep 2005 12:35:16 GMT

You mentioned "the Logical volumes are the same" and obviously both are connected to similar DS2405s. But how sure are you that both are configured the same?

1. How are the logical volumes created on both servers? LVM or VxVM? Stripe or RAID5?

2. How did you test and what gave you the conclusion that the A6826A is "slower"?

3. Can you do compare:

time dd if=/dev/rdsk/cXtYdZ of=/dev/null bs=1024k

on a disk hooked up to the DS2405s on each server?

Re: A6826A Fiber channel slower disk IO then A6795A

Ted Buis — Tue, 27 Sep 2005 08:28:30 GMT

Make sure that the kernel parameter, default_disk_ir is the same on both servers. I have seen up to 4X difference in peformance in write intensive applications. However, consider the risks of having it set to 1.

Re: A6826A Fiber channel slower disk IO then A6795A

Luk Vandenbussche — Tue, 27 Sep 2005 08:47:16 GMT

Ted,

This parameter is not set.
If you have more ideas