topic Re: RHEL4.5 and multipathing to EVA 6100 in Operating System - Linux

RHEL4.5 and multipathing to EVA 6100

nelr — Thu, 17 Apr 2008 03:19:45 GMT

(2.6.9-67.0.7.ELlargesmp x86_64)

Hello everyone,

I have always used the fiberutls and qla drivers from HP to talk to a MSA1500 with active/passive mode. I now have an EVA6100, I understand it's active/active but have no idea what to do on the linux end to make this work properly. I got things working today, however I dont know if it's the correct setup. I was getting 4 luns for every vdisk before I changed this value from 0 to 1. ql2xfailover=1

BEFORE:
scsi0 00 00 00 HP HSV200 6110 RAID
scsi0 00 00 01 HP HSV200 6110 Direct-Access
scsi0 00 01 00 HP HSV200 6110 RAID
scsi0 00 01 01 HP HSV200 6110 Direct-Access
scsi1 00 00 00 HP HSV200 6110 RAID
scsi1 00 00 01 HP HSV200 6110 Direct-Access
scsi1 00 01 00 HP HSV200 6110 RAID
scsi1 00 01 01 HP HSV200 6110 Direct-Access

AFTER:
now, i get one lun per vdisk
(sda 0,0,0,1 HP HSV200 6110)

I know there is the device-mapper-multipath package out there, but do i need it or does the qla work properly?

Thanks in advance
--------------
fibreutils-2.4-1
hp_qla2x00src-8.01.07.25-2
----

options qla2xxx ql2xmaxqdepth=16 qlport_down_retry=30 ql2xloginretrycount=30 ql2xfailover=1 ql2xlbType=0 ql2xautorestore=0xa ConfigRequired=0
remove qla2xxx /sbin/modprobe -r --first-time --ignore-remove qla2xxx && { /sbin/modprobe -r --ignore-remove qla2xxx_conf; }

[root@etc]# adapter_info -v
/proc/scsi/qla2xxx/1:
Driver version = 8.01.07.25-fo
Adapter state = READY
Adapter model = HPAE311A
WWNN = 5001438000c2354d
WWPN = 5001438000c2354c
Queue depth = 16
Port down retry count = 030
Login retry count = 030
/proc/scsi/qla2xxx/0:
Driver version = 8.01.07.25-fo
Adapter state = READY
Adapter model = HPAE311A
WWNN = 5001438000c235f5
WWPN = 5001438000c235f4
Queue depth = 16
Port down retry count = 030
Login retry count = 030
[root@etc]#

Re: RHEL4.5 and multipathing to EVA 6100

slydmin — Thu, 17 Apr 2008 14:53:34 GMT

Yes nelr, there is a package called device-mapper that can be used to solve your multipathing needs.

Briefly, you would need udev to detect hotplugins or path failures. device-mapper-multipath (multipathd) will be useful in providing an alternate link to your disk, should the primary link fail. I think there is way to get a load balancing mode turned on within multipath. Also there is way to forcefully turn of qlogic drivers from performing the multipathing duties. This is probably essential since you may not get desired results with both qlogic and device-mapper having a say about the failed/available links.

So with this setup, you will get a persistent name (/dev/mapper/disk1, say) with to paths to that disk1 /dev/sda , /dev/sdb. I ususally like to partition the LUNs so I see it on fdisk, which prevents wrongful deletion of that disk (fdisk /dev/sda/ --> no parition, probably not being used for anything...train of thought)

Please read my previous post, which has a link for oracle paper.

multipath -l gives a list of devices that currently have a dual link to the LUNs.
eg:
multipath -l
backup_01d (360060e80141a840000011a840000000d)
[size=500 GB][features="0"][hwhandler="0"]
\_ round-robin 0 [active]
\_ 0:0:0:10 sdk 8:160 [active]
\_ 1:0:0:10 sdv 65:80 [active]

-hth,
-s

Re: RHEL4.5 and multipathing to EVA 6100

Steven E. Protter — Thu, 17 Apr 2008 16:47:04 GMT

Shalom,

multipath is built into the OS and nothing further needs to be done to configure it. If the path is there it will get picked up.

I have noticed if there is no configuration on the OS for some time, the OS sort of "forgets" one or both paths to the disk.

the hpscan utility that comes with smart start will renew these connections and restore things to normal.

Configure and leave a filesystem mounted and this problem will not recur.

SEP

Re: RHEL4.5 and multipathing to EVA 6100

Ross Minkov — Thu, 17 Apr 2008 18:08:16 GMT

nelr,

What you did was enable the failover type of multipathing in the QLogic driver -- I see you set ql2xfailover=1 in /etc/modprobe.conf. The fact you saw many paths before and 1 after tell you that it worked. Now your system id aware that all of these paths lead to the same LUN.

Did you also compile the QLogic driver for all valid kernels that it finds:

# cd /opt/hp/src/hp_qla2x00src && ./compile_all_kernels

Did you create a new initrd images for all of the kernels listed in your /boot/grub/grub.conf? (in case someone decides to boot a different kernel)

At a minimum you have to rebuild the initrd for the running kernel, i.e.

# cd /tmp
# mkinitrd -v /tmp/ross.img `uname -r`
# cd /boot
# mv initrd-`uname -r`.img initrd-`uname -r`_ross_nomultipathing.img
# cp /tmp/ross.img /boot/initrd-`uname -r`.img
# ls -la

Take a look at /boot/grub/grub.conf and make sure that the initrd line there calls /initrd-`uname -r`.img (Note that you won't see it this way, with uname -r; it will be more like /initrd-2.6.9-22.ELsmp.img in grub.conf).

For PSP 791 + kernel 2.6.9-55 smp:

cd /boot
mv HP-initrd-2.6.9-55.ELsmp.img HP-initrd-2.6.9-55.ELsmp.img-old
cp initrd-2.6.9-55.ELsmp.img HP-initrd-2.6.9-55.ELsmp.img

Reboot.

After the reboot check that failover multipathing is enabled in the driver using:

# grep 'Driver version' /proc/scsi/qla2xxx/*

Example:

# grep 'Driver version' /proc/scsi/qla2xxx/*
/proc/scsi/qla2xxx/0: Firmware version 4.00.23 [IP] , Driver version 8.01.06.01-fo
/proc/scsi/qla2xxx/1: Firmware version 4.00.23 [IP] , Driver version 8.01.06.01-fo

Note the -fo (for failover) at the end of each driver line. This is what we want to see here. If you don't see it, then you probably did not rebuild your initrd image properly.

HTH,
Ross

Re: RHEL4.5 and multipathing to EVA 6100

nelr — Fri, 18 Apr 2008 02:09:28 GMT

Thank you guys very much for the assistance. I am able to get to my storage via multiple paths. In other words, I can still do I/O when pulling out a fiber from the HBA.

However, my main question is: am i going to be in an active/passive mode with my 2 HBS's or will they be in active/active.

I am attaching both /proc/scsi/qla2xxx outputs, if you look at the bottom under "SCSI Lun information" and "total reqs" you will see that my 0 card is very busy, while my 1 card is doing nothing.

Re: RHEL4.5 and multipathing to EVA 6100

westb — Fri, 18 Apr 2008 12:43:25 GMT

my 2 cents...

I've used rdac and qlogic multipathing on an IBM DS4800 and I find the RHEL multipathing package way easier to maintain and you don't have to deal with issues regarding kernel upgrades; i.e. rebuilding initrd etc., and RedHat won't help you with qlogic issues.

I haven't used it on RHEL4 but has worked for me without problems on RHEL5 xen servers for about 3 months now. An argument can be made that it hasn't been around that long... RHEL4.2 ??.

I'm now using RHEL5 multipathing on both EVA6100 (active/active) and the DS4800 (active/passive); the RHEL multipathing even has an rdac option that I use on the DS4800.