Redhat patching

[AKAT]

We have redhat enterprise 5.2 (Kernel 2.6.18-92.el5) on Blade 685, Our OS Lun is on the disk array(SAN) we do not have OS on external disks. The virtual servers are same and are on VMWARE ESX servers.
We recently patched our virtual linux servers (to 2.6.18-164.2.1.el5) from rhn website (applied all patches) and had no issues with the patching but on our physical servers when the latest redhat patches were installed the server failed to boot and gave below error :
qla2xxx: Unknown parameter 'ql2xfailover'
ismod: error inserting 'lib/qla2xxx.ko' - -1 unknown symbol in module
Kernel Panic

I think the problem is with Qlogic drivers on our physical servers, any idea how we can patch these servers,so far I have tried just applying rhn patches and it fails, I tried patching the servers and then re-installing the qlogic drivers before the reboot and it still failed.
I have also applied the latest proliant pack after the patches are installed and still cannot boot the server.

Our Mezzanine card is - QMH2462 and we are still using the old 8.02 HBA drivers.

FYI - Because the OS is on disk array the server boots via HBA card.

We need to patch our servers and the above error is holding our physical servers to be patched. Please help

I take a snapshot and then rollback so that server is up and available ( this is test server)

Thanks,

[Steven E. Protter]

Shalom,

Boot the server interactively off the previous kernel.

rpm -e the bad kernel.

You may at this point need to boot into rescue mode to save these systems.

SEP

[Matti_Kurkela]

The problem may be that the updated kernel has a "vanilla" version of the Qlogic driver.

The error message "Unknown parameter 'ql2xfailover'" indicates you've been using the add-on driver before. Now the new kernel has the RedHat standard version of qla2xxx which does not have built-in failover features: RedHat would expect you to use the device-mapper-multipath instead of failover features in HBA drivers.

After installing a new kernel but before booting with it, you should do whatever is needed to rebuild the Qlogic driver for the new kernel, then rebuild the new kernel's initrd to ensure that the correct version of Qlogic driver is included to it. The instructions for doing this should come with the add-on Qlogic driver.

MK

[AKAT]

Thanks, I update and submit points once I follow your instructions

[skt_skt]

are you using Powerpath or DM Multipath?

I did recently switch from 2.6.18-53.1.13.el5 to 2.6.18-53.1.21.2.1.el5 and the new kernel did not have the drivers for qla2xx. Since i dont boot from SAN disk, i could reinstall the driver/8.01.07.15 and come up.

Any driver reinstalltion need to be done from(booting) the new kernel.
can u try the mkinitrd after booting from old kernel but pass the arquments relevant to new kernel.
#mkinitrd -f /boot/initrd-$(newkernel).img $(newkernel)

Then update the grub.conf and reboot. (pretty much what MK said)

[Matti_Kurkela]

Santhosh, if the system is booting from SAN, some things are rather different.

> Any driver reinstalltion need to be done from(booting) the new kernel.

If the system boots from SAN and requires the add-on driver to boot, the driver *MUST* be reinstalled/recompiled *before* booting the new kernel. Otherwise the system cannot access the root filesystem on SAN and the boot ends in a kernel panic, just like in AKAT's case.

Last time I looked, both Qlogic and Emulex add-on driver packages had instructions for recompiling the driver module for the new kernel while still running the old one.

The important thing is to be aware that it needs to be done. It does not happen automatically.

(Hmm... looks like the /sbin/new-kernel-pkg script in RHEL 5.x finally includes the possibility to easily plug in custom actions after the installation of a new kernel package.

Apparently all you need is to create the directory /etc/kernel/postinst.d and put in a script that does whatever is necessary. The script will get a full path to the kernel file as parameter $1. I need to test this...)

MK

[Matti_Kurkela]

Just FYI: this is what RedHat has to say about ql2xfailover.

http://kbase.redhat.com/faq/docs/DOC-16994

Not very useful if you wish to keep using Qlogic failover.

MK

[Matti_Kurkela · Accepted solution]

I finally found the instructions to re-compile add-on qla2xxx for a new kernel version from my old notes.

After installing the new kernel, but while still running the old version:

cd /opt/hp/src/hp_qla2x00src
./build_driver.rh -m
mkinitrd -f /boot/initrd-.img

where should be replaced with something like "2.6.18-xxx-y-zz.el5", i.e. whatever the "uname -r" command will be reporting when the new kernel is started.

MK

[AKAT]

Many Thanks, I installed the latest patches and recommended drivers and recomplies as suggested by Matti

[skt_skt]

U have not rewarded points to him yet; U still can

[AKAT]

There you go.... Thanks everyone