RHEL4.4 upgrade to 4.8 mkrootdev label not found

[John Donovan]

Ran up2date -fu on rhel4.4 (2.6.9-22.EL). New versions are rhel4.8 (2.6.9-89.33.1.EL).

Upon reboot, the kernel panics with "mkrootdev: label / not found". Can still boot with old kernel, but not the new.

Search results I've found are for manual kernel builds, so don't think those apply.

fstab is correct and verified correct with e2label.

Can still boot with old kernel, but not the new.

Any thoughts?

Thanks,

[Matti_Kurkela · Accepted solution]

The old kernel still boots, and e2label confirms the label is still OK. So something is preventing the new kernel from reading the label. The most common reason would be the lack of a correct driver.

Does your system require some special driver for your disk/RAID controller that was not included in the standard RHEL 4 distribution? If that's true, the driver is probably installed into the current initrd of the old kernel (only). You will have to do something to add the special driver to the new kernel's initrd too; see the driver documentation for instructions.

If you have a Proliant server with a SmartArray B110i disk controller, be aware that its hpahcisr driver is available in binary form only, for *specific* kernel versions only.

The latest RHEL 4 kernel version supported by the latest hpahcisr driver package is 2.6.9-89.0.26.EL. If you're using SmartArray B110i with RHEL 4, that's the latest kernel version you can currently use. (You would not be the first SmartArray B110i owner that's been bitten by this.)

MK

[John Donovan]

Thanks, Matti...

Did not consider possible issues with hardware. Therefore, left out that this is a VM on VMware ESX4 host.

Additionally, have ran the same update on 20+ other RHEL4.4 servers without incident.

Just baffled why this one failed.

Appreciate your prompt reply...

:-)

[Matti_Kurkela]

On VMware, it's rather easy to capture all the Linux boot messages to a file.

First, add a serial port to the VM configuration, and choose to store the data from the port to a file.

Then power up the VM, interrupt the GRUB bootloader and press E to edit the boot options for the new kernel. Select the "kernel" line, press E again to edit it, remove options "rhgb" and/or "quiet" if they exist, and add "console=tty0 console=ttyS0,115200" instead. This will cause the boot messages to go into the serial port instead of the VM console, so they will be captured into the serial port data file.

Then press Enter, then B to make the VM boot with the changed options. You will see only the kernel messages on the VM console (and because the "quiet" option was removed, there will be a lot of them): the output of the boot scripts will go to the serial port only. Once you see the kernel panic message, you can power down the VM, remove the serial port from the configuration and retrieve the serial port data file.

Examining that file might provide the necessary clues for solving this problem: before the "label / not found" message, there is probably an earlier error message of some kind that normally scrolls off the console before you can see it, or some other vital clue for solving this problem.

MK

[Riaan van Niekerk]

Things to check / try

1) What is the contents of the /etc/modprobe.conf file on the problem VM? (It may have something different from the other VMs in the entry scsi-adapter
2) recreate the initial ramdisk with the command
(rename the original 2.6.9-89.33.1.EL initrd first)
mkinitrd /boot/initrd-2.6.9-89.33.1.EL.img 2.6.9-89.33.1.EL
Compare the "broken" 2.6.9-89.33.1.EL with the new initrd file. If the new one is bigger, it may have included something that was not included when the broken initrd was created.
3) Copy across an initrd from a working system
4) Rerun mkinitrd with -v (verbose) to check which SCSI modules it tries to include.

[John Donovan]

UPDATE: We have since upgraded to RHEL5 (fresh install) for most of the servers, including my "little problem child".
Have not seen the problem since, on any server.
Appreciate everyone's response. If this occurs again, I am better prepared.

Thanks,
:-)