RedHat 4 U7 with QLogic HBAs can't see LUNs on second path

[K. Lesperance]

Hi,

We have 4 Servers, 2 DL 580s and 2 DL 380s, with Qlogic HBAs, connected to an XP 24000. The 580s are running RHEL 4 Update 7 x86_64, the 380s are running RHEL 4 Update 7 i686. The servers are connected to the XP in a dual path configuration, through two separate switches.

Recently, all four servers lost the second path to the LUNs on the HBA. The 2 DL 580s see an ESL library on the port where the LUNs disappeared from, so it doesn't appear to be a cabling issue. The servers all lost the second path at the same time.

If I cat /proc/scsi/qla2xxx/0 and /proc/scsi/qla2xxx/1, I notice one of the targets is missing:

qla0:

SCSI Device Information:
scsi-qla0-adapter-node=5001438002000b85;
scsi-qla0-adapter-port=5001438002000b84;
scsi-qla0-target-0=100000e00222cc74;
scsi-qla0-target-1=100000e00242cfd4;

FC Port Information:
scsi-qla0-port-0=50060e8005b1cb21:50060e8005b1cb21:022600:1000;
scsi-qla0-port-1=100000e00202cc74:100000e00222cc74:028e00:82;
scsi-qla0-port-2=100000e00202cfd4:100000e00242cfd4:020e00:83;

qla1:

SCSI Device Information:
scsi-qla1-adapter-node=5001438002000b87;
scsi-qla1-adapter-port=5001438002000b86;
scsi-qla1-target-0=50060e8005b1cb31;
scsi-qla1-target-1=100000e00242cc74;
scsi-qla1-target-2=100000e00222cfd4;

FC Port Information:
scsi-qla1-port-0=50060e8005b1cb31:50060e8005b1cb31:022600:81;
scsi-qla1-port-1=100000e00202cc74:100000e00242cc74:028b00:82;
scsi-qla1-port-2=100000e00202cfd4:100000e00222cfd4:020e00:83;

I'm wondering if the missing target is why we're seeing the library but not the disks, what could be causing the target to disappear, and how we can get it back. I suspect it is a zoning issue, but our SAN team insists it is not.

Any help in this matter would be much appreciated.

Thanks

[Steven E. Protter]

Shalom,

Is PSP installed?

what does hp_rescan utility show. If its a zoning issue this will be apparent once hp_rescan -a is run.

Linux has a default multipath configuration built into the kernel.

I suspect wiring is at issue?

Was initrd modified to not include qla drivers?

Did this work on a prior release of RHEL 4.X

SEP

[Simon Hargrave]

When running the qlogic driver in failover mode it's quite normal to see no luns on the second interface /proc/scsi/qla2xxx/1. This is so that the SCSI layer doesn't see the same device twice and give you sda and sdb for the same LUN - it's all handled by the SCSI driver.

Assuming you're using the HP driver bundle rather than the RHEL provided drivers (if not, why not?) then you can: -

cat /etc/modprobe.conf and confirm that ql2xfailover=1 is set. That will confirm that you're using the driver in failover mode and why you only see the LUN down qla2xxxx/0

If you want to be sure you have connectivity down both paths, then there are a number of options depending on how destructive to downtime you can be: -

1. The sure way is to reconfigure the driver to single-path. Change directory to /opt/hp/src/hp_qla2x00src and run set_parm and choose Single Path. Let it build initrd and reboot. You should then see your LUNs down qla2xxx/0 and /1. Rerun set_parm and choose XP Failover and again reboot.

2. If you have e.g. Cisco SAN switches then login to your device manager and look at the traffic down the paths. You'll see traffic down one of the ports, then disable/unplug that port and verify that you get data down the second.

[Kevin Cauchi]

I work with Kelly - If we run /opt/hp/hp_fiberutils/sysfs_scandisk 0/1 should we see the LUNS on both paths? When we run an hp_rescan we don't seem to be picking up LUNs on one of the paths. We are however picking up tape devices on both paths (as per output from sysfs_scandisk. See output below:

[root@rsgisdp1 hp_fibreutils]# ./sysfs_scandisk 0
0:0:0:0 HP ESL9000 Series
0:0:1:0 HP Ultrium 2-SCSI
0:0:1:1 HP Ultrium 2-SCSI
[root@rsgisdp1 hp_fibreutils]# ./sysfs_scandisk 1
1:0:0:0 HP Ultrium 2-SCSI
1:0:0:1 HP Ultrium 2-SCSI
1:0:1:0 HP Ultrium 2-SCSI
1:0:1:1 HP Ultrium 2-SCSI
1:0:2:0 HP OPEN-V
1:0:2:1 HP OPEN-V
1:0:2:10 HP OPEN-V
1:0:2:11 HP OPEN-V
1:0:2:12 HP OPEN-V
1:0:2:13 HP OPEN-V
1:0:2:14 HP OPEN-V
1:0:2:15 HP OPEN-V
1:0:2:16 HP OPEN-V
1:0:2:17 HP OPEN-V
1:0:2:18 HP OPEN-V
1:0:2:19 HP OPEN-V
1:0:2:2 HP OPEN-V
1:0:2:20 HP OPEN-V
1:0:2:21 HP OPEN-V
1:0:2:22 HP OPEN-V
1:0:2:23 HP OPEN-V
1:0:2:24 HP OPEN-V
1:0:2:25 HP OPEN-V
1:0:2:26 HP OPEN-V
1:0:2:27 HP OPEN-V
1:0:2:28 HP OPEN-V
1:0:2:29 HP OPEN-V
1:0:2:3 HP OPEN-V
1:0:2:30 HP OPEN-V
1:0:2:31 HP OPEN-V
1:0:2:32 HP OPEN-V
1:0:2:33 HP OPEN-V
1:0:2:34 HP OPEN-V
1:0:2:35 HP OPEN-V
1:0:2:36 HP OPEN-V
1:0:2:37 HP OPEN-V
1:0:2:38 HP OPEN-V
1:0:2:39 HP OPEN-V
1:0:2:4 HP OPEN-V
1:0:2:40 HP OPEN-V
1:0:2:41 HP OPEN-V
1:0:2:42 HP OPEN-V
1:0:2:43 HP OPEN-V
1:0:2:5 HP OPEN-V
1:0:2:6 HP OPEN-V
1:0:2:7 HP OPEN-V
1:0:2:8 HP OPEN-V
1:0:2:9 HP OPEN-V
[root@rsgisdp1 hp_fibreutils]# ./hp_rescan -a
Sending rescan signal to /proc/scsi/qla2xxx/0...
Sending rescan signal to /proc/scsi/qla2xxx/1...
Adding legacy tape devices to /proc/scsi/device_info
Scanning /proc/scsi/qla2xxx/0, target 1, LUN 1
Scanning /proc/scsi/qla2xxx/1, target 2, LUN 43


scsi0 00 00 00 HP ESL9000 3.60 Medium
scsi0 00 01 00 HP Ultrium F6AW Sequential-Access
scsi0 00 01 01 HP Ultrium F6AW Sequential-Access
scsi0 00 00 01 HP Ultrium F6AW Sequential-Access
scsi0 00 00 02 HP Ultrium F6AW Sequential-Access
scsi1 00 00 00 HP Ultrium F6AW Sequential-Access
scsi1 00 00 01 HP Ultrium F6AW Sequential-Access
scsi1 00 01 00 HP Ultrium F6AW Sequential-Access
scsi1 00 01 01 HP Ultrium F6AW Sequential-Access
scsi1 00 02 00 HP OPEN-V 6003 Direct-Access
scsi1 00 02 01 HP OPEN-V 6003 Direct-Access
scsi1 00 02 02 HP OPEN-V 6003 Direct-Access
scsi1 00 02 03 HP OPEN-V 6003 Direct-Access
scsi1 00 02 04 HP OPEN-V 6003 Direct-Access
scsi1 00 02 05 HP OPEN-V 6003 Direct-Access
scsi1 00 02 06 HP OPEN-V 6003 Direct-Access
scsi1 00 02 07 HP OPEN-V 6003 Direct-Access
scsi1 00 02 08 HP OPEN-V 6003 Direct-Access
scsi1 00 02 09 HP OPEN-V 6003 Direct-Access
scsi1 00 02 10 HP OPEN-V 6003 Direct-Access
scsi1 00 02 11 HP OPEN-V 6003 Direct-Access
scsi1 00 02 12 HP OPEN-V 6003 Direct-Access
scsi1 00 02 13 HP OPEN-V 6003 Direct-Access
scsi1 00 02 14 HP OPEN-V 6003 Direct-Access
scsi1 00 02 15 HP OPEN-V 6003 Direct-Access
scsi1 00 02 16 HP OPEN-V 6003 Direct-Access
scsi1 00 02 17 HP OPEN-V 6003 Direct-Access
scsi1 00 02 18 HP OPEN-V 6003 Direct-Access
scsi1 00 02 19 HP OPEN-V 6003 Direct-Access
scsi1 00 02 20 HP OPEN-V 6003 Direct-Access
scsi1 00 02 21 HP OPEN-V 6003 Direct-Access
scsi1 00 02 22 HP OPEN-V 6003 Direct-Access
scsi1 00 02 23 HP OPEN-V 6003 Direct-Access
scsi1 00 02 24 HP OPEN-V 6003 Direct-Access
scsi1 00 02 25 HP OPEN-V 6003 Direct-Access
scsi1 00 02 26 HP OPEN-V 6003 Direct-Access
scsi1 00 02 27 HP OPEN-V 6003 Direct-Access
scsi1 00 02 28 HP OPEN-V 6003 Direct-Access
scsi1 00 02 29 HP OPEN-V 6003 Direct-Access
scsi1 00 02 30 HP OPEN-V 6003 Direct-Access
scsi1 00 02 31 HP OPEN-V 6003 Direct-Access
scsi1 00 02 32 HP OPEN-V 6003 Direct-Access
scsi1 00 02 33 HP OPEN-V 6003 Direct-Access
scsi1 00 02 34 HP OPEN-V 6003 Direct-Access
scsi1 00 02 35 HP OPEN-V 6003 Direct-Access
scsi1 00 02 36 HP OPEN-V 6003 Direct-Access
scsi1 00 02 37 HP OPEN-V 6003 Direct-Access
scsi1 00 02 38 HP OPEN-V 6003 Direct-Access
scsi1 00 02 39 HP OPEN-V 6003 Direct-Access
scsi1 00 02 40 HP OPEN-V 6003 Direct-Access
scsi1 00 02 41 HP OPEN-V 6003 Direct-Access
scsi1 00 02 42 HP OPEN-V 6003 Direct-Access
scsi1 00 02 43 HP OPEN-V 6003 Direct-Access

[K. Lesperance]

Steven,

When I run hp_rescan, it only seems to scan up to lun1 on qla0:

Scanning /proc/scsi/qla2xxx/0, target 1, LUN 1
Scanning /proc/scsi/qla2xxx/1, target 2, LUN 43


scsi0 00 00 00 HP ESL9000 3.60 Medium
scsi0 00 01 00 HP Ultrium F6AW Sequential-Access
scsi0 00 01 01 HP Ultrium F6AW Sequential-Access
scsi0 00 00 01 HP Ultrium F6AW Sequential-Access
scsi0 00 00 02 HP Ultrium F6AW Sequential-Access
scsi1 00 00 00 HP Ultrium F6AW Sequential-Access
scsi1 00 00 01 HP Ultrium F6AW Sequential-Access
scsi1 00 01 00 HP Ultrium F6AW Sequential-Access
scsi1 00 01 01 HP Ultrium F6AW Sequential-Access
scsi1 00 02 00 HP OPEN-V 6003 Direct-Access
scsi1 00 02 01 HP OPEN-V 6003 Direct-Access
scsi1 00 02 02 HP OPEN-V 6003 Direct-Access
scsi1 00 02 03 HP OPEN-V 6003 Direct-Access
scsi1 00 02 04 HP OPEN-V 6003 Direct-Access
scsi1 00 02 05 HP OPEN-V 6003 Direct-Access
scsi1 00 02 06 HP OPEN-V 6003 Direct-Access
scsi1 00 02 07 HP OPEN-V 6003 Direct-Access
scsi1 00 02 08 HP OPEN-V 6003 Direct-Access
scsi1 00 02 09 HP OPEN-V 6003 Direct-Access
scsi1 00 02 10 HP OPEN-V 6003 Direct-Access
scsi1 00 02 11 HP OPEN-V 6003 Direct-Access
scsi1 00 02 12 HP OPEN-V 6003 Direct-Access
scsi1 00 02 13 HP OPEN-V 6003 Direct-Access
scsi1 00 02 14 HP OPEN-V 6003 Direct-Access
scsi1 00 02 15 HP OPEN-V 6003 Direct-Access
scsi1 00 02 16 HP OPEN-V 6003 Direct-Access
scsi1 00 02 17 HP OPEN-V 6003 Direct-Access
scsi1 00 02 18 HP OPEN-V 6003 Direct-Access
scsi1 00 02 19 HP OPEN-V 6003 Direct-Access
scsi1 00 02 20 HP OPEN-V 6003 Direct-Access
scsi1 00 02 21 HP OPEN-V 6003 Direct-Access
scsi1 00 02 22 HP OPEN-V 6003 Direct-Access
scsi1 00 02 23 HP OPEN-V 6003 Direct-Access
scsi1 00 02 24 HP OPEN-V 6003 Direct-Access
scsi1 00 02 25 HP OPEN-V 6003 Direct-Access
scsi1 00 02 26 HP OPEN-V 6003 Direct-Access
scsi1 00 02 27 HP OPEN-V 6003 Direct-Access
scsi1 00 02 28 HP OPEN-V 6003 Direct-Access
scsi1 00 02 29 HP OPEN-V 6003 Direct-Access
scsi1 00 02 30 HP OPEN-V 6003 Direct-Access
scsi1 00 02 31 HP OPEN-V 6003 Direct-Access
scsi1 00 02 32 HP OPEN-V 6003 Direct-Access
scsi1 00 02 33 HP OPEN-V 6003 Direct-Access
scsi1 00 02 34 HP OPEN-V 6003 Direct-Access
scsi1 00 02 35 HP OPEN-V 6003 Direct-Access
scsi1 00 02 36 HP OPEN-V 6003 Direct-Access
scsi1 00 02 37 HP OPEN-V 6003 Direct-Access
scsi1 00 02 38 HP OPEN-V 6003 Direct-Access
scsi1 00 02 39 HP OPEN-V 6003 Direct-Access
scsi1 00 02 40 HP OPEN-V 6003 Direct-Access
scsi1 00 02 41 HP OPEN-V 6003 Direct-Access
scsi1 00 02 42 HP OPEN-V 6003 Direct-Access
scsi1 00 02 43 HP OPEN-V 6003 Direct-Access

On 2 of the hosts we were initially using the RedHat multipath software (multipathd). Not long before the problem started to occur we installed the HPDM multipath software. On the other 2 hosts, we're using Veritas for multipathing.

The drivers are being loaded, we've tried with and without failover enabled.

We haven't used a prior RedHat version, this is a new environment.

Simon,

We've tried the qlogic driver with failover enabled and disabled, we see the same thing each time. We're using the HP Qlogic drivers.

[K. Lesperance]

Everyone, thanks for your help. Turns out it was a problem with the switch. We replaced the GBIC, and everything is working fine now.

[K. Lesperance]

Closing thread per previous comments.