RHEL 4 QLogic driver update problem

[Leibniz]

Hello everybody:

I have another problem on which I hope someone can shed some light...

Environment
ProLiant DL580 G3
RHEL 4.8
kernel 2.6.9-89.ELsmp #1 SMP Mon Apr 20 10:34:33 EDT 2009 i686 i686 i386 GNU/Linux
QLogic fibre card: QLA2342
EMC with PowerPath v. 5.3

While running an emcgrab report a few weeks ago, we found this problem:

...
Driver Version 8.01.07.15-fo
EMC Supported Driver version(s) 8.02.09.02.04.08-d
WARNING Detected version (8.01.07.15-fo) does not match any of the EMC supported Driver version(s) listed
...

Therefore, I attempted to update the driver to 8.02

In the process, the only 8.02 driver I found from the QLogic website, was this one: 8.02.09-d0-rhel4.7-05

As that driver archive doesn't have an install script (qinstall) - I went the manual method - extracted the files from the archive, and ran the build.sh script. Copied the .ko modules over to the /lib/modules directory, and then ran:
mkinitrd -f initrd..img

Well sir, when I rebooted it dropped into maintenance mode because it could not find any SAN located filesystems from the /etc/fstab file. (Errored on one fs, I booted to single user, removed it from fstab, and then rebooted - only to have it crap out on the next fs.)

I restored my saved initrd.img and rebooted, and then everything came back.

In the mean time, the new qla driver modules are still in the /lib/modules tree. (didn't restore the old qla kernel modules to the tree.)

Now, what I think I have is a system that boots using qla driver 8.01.07.15-fo to load all the SAN partitions - and then later in the boot process, loads the newer module that I manually copied into the /lib/modules tree.

I assume that because of the fact that I restored the initrd.img (which had the older driver) plus the fact that modinfo tells me I'm using the newer driver:

# modinfo qla2xxx | grep version
version: 8.02.09-d0-rhel4.7-05

So the question is this: what did I miss while creating a new initrd.img which caused non of the SAN VGs to be seen on boot?

Is there a more complex mkinitrd process that requires info about lvm and or SAN? I've not heard of such, and I've read countless readme files on creating an initrd img - including the one in the driver software archive, and nothing indicates that I need to pass any other parms to the mkinitrd.

The only change I made was to build and manually copy the .ko files into the /lib/modules tree (as per manual) and then run mkinitrd, and all that followed was major problems.

Can anyone lend any insight?

Thanks

[Steven E. Protter]

Shalom,

Insight? No.

So the question is this: what did I miss while creating a new initrd.img which caused non of the SAN VGs to be seen on boot?

Driver incompatibility with your kernel or the way you hacked this driver into use.

Is there a more complex mkinitrd process that requires info about lvm and or SAN? I've not heard of such, and I've read countless readme files on creating an initrd img - including the one in the driver software archive, and nothing indicates that I need to pass any other parms to the mkinitrd.

No, mkinitrd on all currently released Linux OS does not support LVM boot.

SEP

[Leibniz]

And when you say "hack" I assume you mean "followed the install directions carefully", i.e.

# cd 8.02.09-d0-rhel4.7-05/
# cat readme_qla2xxx_inbox.txt
---snip---
4. Build the driver modules from the source code by starting the
build.sh script. For example:

# ./build.sh

--snip- To load drivers manually... ---

* To load the driver using modprobe:

1. Copy the .ko files to the appropriate
/lib/modules/2.6.../kernel/drivers/scsi/qla2xxx directory.
For example:

# cp *.ko /lib/modules/2.6.../kernel/drivers/scsi/qla2xxx

2. Enter the following commands:

# modprobe -v qla2300 (QLA23XX) or
# modprobe -v qla2400 (QLE24XX/QLE24XX/QLE8042) or
# modprobe -v qla2500 (QLE25XX)

NOTE: The modprobe -v qla2300/qla2400/qla2500 command
automatically loads the qla2xxx.ko and
scsi_transport_fc.ko components.

* To unload the driver using modprobe, enter the following
command:

# modprobe -r qla2300/qla2400/qla2500

This command unloads the qla2300.ko and qla2xxx.ko modules.
If the system finds additional firmware-loader modules (such as
qla2400.ko) that depend on qla2xxx.ko, it will not unload
the qla2xxx.ko file.

5.3 Making a RAMDISK Image to Load the Driver

Use the following steps to build a ramdisk image on RHEL 4.7:

1. Follow the steps in section 5.1

2. Copy the .ko files to the appropriate
/lib/modules/2.6.../kernel/drivers/scsi/qla2xxx
directory. For example:

# cp *.ko /lib/modules/2.6.../kernel/drivers/scsi/qla2xxx

3. Edit the /etc/modprobe.conf file and add the following entries
(if not already present):

alias scsi_hostadapter1 qla2xxx
alias scsi_hostadapter2 qla2300 (QLA23XX) or
alias scsi_hostadapter3 qla2400 (QLA24XX/QLE24XX/QLE8042) or
alias scsi_hostadapter4 qla2500 (QLE25xx)

4. Change to the /boot directory for x86, and x86_64 platforms, and
to /boot/efi/efi/redhat for ia64 platforms.

5. Create a backup of current RAMDISK image. For example:

# cp -f initrd-[kernel_version].img initrd-[kernel_version].img.bak

6. Build the RAMDISK image by executing the following command:

# mkinitrd -f initrd-[kernel_version].img [kernel_version]

7. Reboot the system to load the RAMDISK image with the QLogic driver.
---snip---

And voila! can't find SAN partitions and drops into maintenance mode.

A driver incompatability does make sense for the problem. If the QLA driver fails in the initrd while booting, it follows that it would not find any SAN partitions.

That said - version 8.02.09.02.04.08-d seems to be the EMC required version of the driver, and yet that specific version does not seem to be available on the QLogic download site.

Anyone know if that version is maybe a non-QLogic version of the driver (EMC? RH?)

Thanks.

[Matti_Kurkela · Accepted solution]

At some point(s) of the RHEL 4 release cycle, the built-in QLogic driver was updated.

The qla2xxx driver version 8.02.09.01.04.09-d is currently included in the standard RHEL 4 kernel package (version 2.6.9-89.0.26.ELsmp), so you don't have to install any extra driver packages from HP nor from QLogic. Just update your kernel to the latest version available from RHN.

If your system uses the older module to mount the SAN partitions, the newer module cannot replace it: you cannot unload the driver module while the SAN filesystems are mounted. The module loading command knows that a module with the same name is already loaded and in use, so it does nothing at all.

The modinfo command simply looks at the module file in your /lib/modules tree, so you're really still using the older module included in the initrd.img, and just fooling the modinfo command with the newer module file.

To find the version number of the qla2xxx module you're really using, run:

head /proc/scsi/qla2xxx/*

If you want to use the module version 8.02.09.01.04.09-d, verify your /etc/modprobe.conf file does not contain any options for the qla2xxx module that the new version does not understand: unrecognized options may stop the module from loading.

Another thing you should know: if your /etc/modprobe.conf contains more than one "options qla2xxx ..." line, the initrd.img creation process may become confused and create a broken early-startup script within the initrd.img file.

I once saw such a case (on about RHEL 4.4, I think) and the system was stuck in an infinite loop within the initrd, spewing lots messages on the console. I had to set up my laptop as a serial console to capture the earliest error messages in order to get to the bottom of that mess...

MK

[Leibniz]

Thanks for your response.

Upon further investigation, what we have is an EMC supported kernel version ( 2.6.9-89.ELsmp) which, for some reason, had an old version of the QLA driver in it (8.01.07.15-fo) and I just learned what you also posted - that the [supported and proper] QLA driver is already compiled in the kernel.

So, we still don't know how that old module got in there, but afte we boot of an older kernel and then force a kernel upgrade, we expect to get the problem resolved.

Thanks again for your help.
Bill