GRUB multiboot with redhat ES and DL380

[Bill McNAMARA_1]

Hi,

I have DL380s G4 and G5's with multiple Raid 0 disks.

Each disk has a seperate OS installed (all redhat).

I want to configure GRUB to offer a choice of which disk to boot from on startup if possible..

or would this be done via the bios.

Are there any considerations with OS installation that I have to take into account..

for example... The plan is to have all disks inserted.. redhat/bios settings will probably call disk in slot 0 hda, slot 1 hdb etc.. will this create impact for reinstalling linux on hdb later on?
Where should grub reside?
What should be configured at the CCISS part...

At the moment, to boot from each disk, I remove all others and am left with a single hda.

Thanks,
Bill

[Stuart Browne]

Hrm.

Interesting problem.

If the goal is to leave all disks in, in a fixed order, then it's definately doable.

If the order changes, or more precisely the first-disk changes, then it makes life very difficult.

You can easily set grub up to boot from the different devices from the same grub config file.

What's probably best is to do the following two things:

In each individual OS, install Grub into the first partition (as against the boot-sector).

This will allow you to have individual (and per-os changes) to the kernel's for updates, if you want that. If not, then skip this bit.

Next is to have the first disks (hopefully the permenant disk) Grub point to each separate disk's first-partition, and 'chainloader' them.

i.e.

title 2nd disk
hd(1,0)
chainloader +1

title 3rd disk
hd(2,0)
chainloader +1

The chain-loader will then ask the device listed (hd(x,0)) for a boot-loader.

It's basically how you multi-boot windows/linux with Grub being the master.

It *WILL* get messy..

Anybody else got a better way? ;)

[Bill McNAMARA_1]

I'm wondering if the map option will help......

for example the windows xp version below changes drive number :

#boot=/dev/hda
default=1
timeout=7
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title Red Hat Enterprise Linux AS (2.4.21-20.EL)
root (hd0,0)
kernel /vmlinuz-2.4.21-20.EL ro root=LABEL=/
initrd /initrd-2.4.21-20.EL.img
title Windows XP
root (hd1,0)
map (hd1) (hd0)
map (hd0) (hd1)
makeactive
chainloader +1

[Ivan Ferreira]

I think that is not that simple, I had problems in the past with this because normally, nowdays, grub and fstab uses devices LABELS to mount the file systems.

As the different Linux installations wil all have the same label, then you will always mount the first file system with the matchel label.

It's a very extrange configuration, with virtualization technologies, you souldn't need booting from different disks. Just start another os instance as a guest.

[Steven E. Protter]

Shalom,

Here is how I did it for what its worth.

I had a Windows system on disk 1, nothing on Disk 2.

I installed Linux on disk 2 and let grub be the boot manager for the entire system.

I acheived results that I believe are similar to what you want.

Doing the windows first is the key so it doesn't stomp on grub later.

The grub records will end up on the primary boot disk.

SEP

[Bill McNAMARA_1]

Hi Steven,

Try installing linux on a third disk with GRUB now...

and then on a fourth!

Its easy enough with different OS's but I haven't managed GRUBbing with the same OS. (linux)

Thanks..
Bill

[Bill McNAMARA_1]

any new ideas here? internet searches are not even helping...

I've tried the following, but just can't have more that one redhat install per server.
I want to have 4 raid 0 redhat installs each selectable via grub.

Currently the only way i have is to remove all disks and to install seperately. when I want to boot a different redhat disk, it's a case of travel 100 miles to change a disk...

Bill

[Arnd Kohlen]

Hm, another suggestion is to put grub to a place which doesn't change if you remove or change any of the hard discs: on a floppy disc or usb stick, cd... Even if the saved configuration doesn't work anymore, you can edit the options at boot time (with the "e"-key) to get one of your remaining harddiscs started.

Later you can use udev to identify your harddiscs or partitions to mount them to the appropriate directories so you don't depend on "sda" or "sdb".

But maybe I'm blind and I can't see the real problem ;-)

Regards
Arnd

[Steven E. Protter]

Shalom,

>>
Try installing linux on a third disk with GRUB now...

and then on a fourth!

Its easy enough with different OS's but I haven't managed GRUBbing with the same OS. (linux)
<<

There is a stage in the GUI installation where you can tell the installation what disk to install the MBR/PBR and grub. It should pick up the previous configuration and afterwards present boot options for multiple Linuxes.

I don't have any systems here with three eligible boot disks to just try it out for you.

If the OS can't do it, then I'd go for the CISS/BIOS route, but I think the OS should be able to do this, unless its stepping on itself.

SEP

[Bill McNAMARA_1]

ok, I think I haven't been clear.

I have 4 disks of raid 0 in a proliant dl380.

+-----+-----+-----+-----+
| | | | |
| | | | |
| | | | |
+-----+-----+-----+-----+

when I install one redhat I get this:
+-----+-----+-----+-----+
| | | |rhel3|
| | | |c0d0 |
| | | | |
+-----+-----+-----+-----+

I can reboot the os no problem, grub is the boot loader.

I try to install another disk:
+-----+-----+-----+-----+
| | |rhel3|rhel3|
| | |c0d1 |c0d0 |
| | | | |
+-----+-----+-----+-----+

I cannot now boot from c0d1.

The only os that is available to boot from is d0.

I want to be able to have grub
on all disks proposing a choice of booting
from any disk c0d0 c0d1, c0d2, c0d3

Bill

[Matti_Kurkela · Accepted solution]

You'll have to read the GRUB documentation and do some customizing. The BIOS is so restricted by PC legacy that using it to control multi-booting tends to be rather painful.

First, GRUB gets all its knowledge of disks from the BIOS. When a computer is booting, the BIOS will make a list of all the disks it can access. The BIOS will identify each hard disk by a hexadecimal number, starting from 0x80 (the numbers of the floppy disks start from 0x00). This numbering is created dynamically each time system boots up: if you add or remove disks, the numbers of the other disks may/will change.

How does this relate to GRUB's disk identifiers? Well, if you don't confuse the situation with "map" directives, BIOS's 0x80 = GRUB's (hd0), 0x81 = (hd1) and so on. With Linux, you should not need any "map" directives at all.

If the BIOS has a facility to select a disk to boot from, that facility usually works by making that disk become the disk 0x80 in the BIOS numbering, and thus (hd0) for GRUB. What happens to the numbers of the other disks is "implementation-specific" - i.e. a likely cause for headaches: this is why I don't recommend using BIOS settings to handle complex multi-boot situations.

The BIOS loads the Master Boot Record from disk 0x80, i.e. the first hard disk on its list. The Master Boot Record might contain the boot loader (the usual way with GRUB), or just a simple program to load the real boot loader from the beginning of the active partition of the same disk (the Microsoft way, although GRUB can be used like this too).

If the boot loader needs to load some more parts of itself (like GRUB's stage2), it usually expects to find them on BIOS disk 0x80, unless specifically configured otherwise. Because the Master Boot Record is always read from disk 0x80, there is not much point to place the bootloader to any other disk: it would just increase the possibilities for failure.

When the Linux kernel starts up, it re-identifies all the disks. There is no general way to ensure that the detection order of Linux matches the detection order of the BIOS if you have multiple controllers, but if you have just a single SmartArray controller you can usually trust that
if disk slot 1 = 0x80 = (hd0) = /dev/cciss/c0d0,
then disk slot 2 = 0x81 = (hd1) = /dev/cciss/c0d1
and so on.
If you have multiple disk controllers, the problem may become a lot more challenging.

When you get this figured out, you can start writing extra entries in the GRUB configuration file. For the simplicity's sake, I recommend that you make the first disk, i.e. /dev/cciss/c0d0 as your "master" bootloader disk. (Otherwise you'll need to get your BIOS make some other disk get the number 0x80, which makes the entire numbering scheme more difficult to predict... see above.)

The /boot/grub/grub.conf (or in some distributions, /boot/grub/menu.lst) file will contain the GRUB boot configurations for all the OSes you have in this system.

Look at the entries for your first OS (c0d0) in the grub.conf file. They should be similar to this:

title
root (hd0,0)
kernel /vmlinuz. root=
initrd /initrd.img.

The "title" line begins each boot menu item definition and defines the text to display in the boot menu.

The "root" line is a bit of a misnomer: it has nothing at all to do with any root filesystem. It just identifies the partition (=filesystem) the next lines will refer to. In a typical Linux installation, it will refer either to the root filesystem or the /boot filesystem.

Note: if /boot is a separate filesystem, the file paths on the "kernel" and "initrd" lines will not have /boot prefixed to them, because GRUB is just looking at the detached /boot filesystem, not the complete tree seen by Linux.

The "kernel" and "initrd" lines will identify the files GRUB will load into the system memory whenever this menu item is selected. On the "kernel" line, you can also specify Linux kernel options, of which the "root=" option is probably the most important one here. RedHat normally uses "root=LABEL" or "root=UUID" style root definitions, but nothing is preventing you from using a simple "root=/dev/cciss/c0d1" here.

To get the correct path names for writing the boot entry for e.g. c0d1 when running the OS of c0d0, just manually mount the necessary partition(s) of c0d1 to /mnt and take a peek.
The GRUB entry to boot from c0d1 should look something like this:

title "boot from (hd1)"
root (hd1,0)
kernel /vmlinuz root=/dev/cciss/c0d1
initrd /initrd.img

To help you to identify the correct GRUB disk names, you could create an uniquely-named file to each installation's root directory (or /boot directory, if /boot is a separate filesystem - that is what we want to find when booting a system). You could then put GRUB in command prompt mode and use its "find" command to find out the correct GRUB disk identifier. (It's sad GRUB does not have a "ls" command.)

For example, if you're running the OS of c0d0 and have manually mounted the /boot partition of the c0d1 OS to /mnt:

touch /mnt/system_c0d1
umount /mnt
shutdown -r now

find /system_c0d1

GRUB should respond with something like:

(hd1,0) /system_c0d1

Now you know for sure that GRUB sees the /boot partition of the c0d1 installation currently as (hd1,0). If you move disks around, the number might change.

Because only the c0d0 installation's GRUB configuration is actually meaningful, you'll probably need to update it manually whenever you update the kernel on any of the other installations. The alternative is to replace the standard boot configuration auto-update system with a script you make yourself: if you update the kernels in this system frequently, it might be worth the effort... or not.

MK

[Marko Krstic]

Hello Bill,

My opinion is that this shouldn't be that complicated. The logic should be very similar to the situation when You have Windows and Linux on the same HDD and GRUB gives You the possibility to choose among them.

When You install the first RH(RH1) on the first HD (HD1), the GRUB is installed there. When You install second RH2 on HD2, You can boot only RH1 (RH2 installed it's grub in MBR of HD2). So, You need to change the menu.lst file on the HD1 and to tell him that You have another OS on another HD (HD2). This way, You'll have unique GRUB file, allways on the fisrt disk, and every time You reboot a machine, this file will be checked.



title RH1
root (hd0,2)
kernel
initrd /boot/initrd.img-2.6.20-16-generic
quiet
savedefault

title RH2

# You should put here another disk,
# ie. (hd1,2)
root (hd1,2)

kernel
initrd /boot/initrd.img-2.6.20-16-generic



For the details about naming hard disks in GRUB, check the manual :-)

When You make changes to menu.lst file, after reboot You should be able to see the GRUB menu with available OS and You should see something like this:

RH1
RH2

See You soon in Belgrade ;-)

Best Regards,
Marko