rpmbuild against different kernels

[Ryan Hobbs_2]

I am fairly new to RH. I'm trying to compile the bcm5700 network driver for an HP DL385 for all installed kernels (against appropriate kernel sources).

Currently kernel installed is 2.6.9-22.0.2.ELsmp. I just ran up2date -quf and updated the kernel to the latest version (kernel-smp-2.6.9-34.EL & kernel-2.6.9-34.EL). I have the bcm5700 driver compiled for the current running kernel. However when I reboot, I am assuming it will be detected as the previous tg3 and since I have tg3 in the /etc/hotplug/blacklist it won't get loaded and my NIC will fail.

# rpm -qa | grep kernel
kernel-utils-2.4-13.1.69
kernel-2.6.9-22.EL
kernel-2.6.9-22.0.2.EL
kernel-2.6.9-34.EL
kernel-smp-2.6.9-22.EL
kernel-smp-2.6.9-22.0.2.EL
kernel-smp-2.6.9-34.EL
kernel-smp-devel-2.6.9-22.0.2.EL (had to manually re-install after
up2date was run)
kernel-smp-devel-2.6.9-34.EL

The HP installation document reads:
"The bcm5700 driver requires the presence of the kernel source code and configuring the kernel source before building the bcm5700 driver. The following steps need to be done once for each kernel that is booted. For example, if the current kernel is UP (uni-processor) and an SMP (symmetrical-multi-processor) kernel is booted, these steps must be performed again to configure the kernel source forSMP before building the bcm5700 driver for the SMP kernel."

I understand all of this. My questions are:

1) Is there a problem having multiple kernel sources installed on the machine? I would assume it would be preferrable to have all sources for all installed kernels.

2) When I ran 'up2date -puf' my previous kernel remained installed (which I understand if removed would crash the running system), whereas when kernel-devel was updated by up2date, it removed the old version and then installed the new version. Why is that? How can it be left alone in order to keep the kernel source for the kernels that are installed?

3) Is there a way one can build a module against a specific kernel source while running another kernel? I.e. running kernel "A" but want to compile against /usr/src/kernels/B/ ? If so, how do you do this? When I performed rpmbuild -bb SPECS/bcm5700.spec I was running kernel-smp-2.6.9-22.0.2.EL but the kernel source for that had been replaced with the source: kernel-smp-devel-2.6.9-34.EL by up2date and the rpmbuild failed. Basically compile/build drivers against multiple kernels without having to reboot into each specific kernel.

4) How many kernels should I leave installed on my machine? I am assuming it would be prudent to only have 2 or 3 versions (with their UP counterpart for a total of 4-6)??

Thanks in advance for the help.

[Steven E. Protter]

Shalom,

1) Is there a problem having multiple kernel sources installed on the machine? I would assume it would be preferrable to have all sources for all installed kernels.

No. Support for tg3 or bcm5700 is built into all of the above kernels. You should not have to do anything. I've found this out through real experience.

2) When I ran 'up2date -puf' my previous kernel remained installed (which I understand if removed would crash the running system), whereas when kernel-devel was updated by up2date, it removed the old version and then installed the new version. Why is that? How can it be left alone in order to keep the kernel source for the kernels that are installed?

You don't need to do any compiles to get the bcm5700 driver to work and you are wrong on your kernel assumptions. The latest kernel broke clustering on my boxes, but you can have as many kernels as you want on a system. Removing an older kernel with rpm is permitted and should damage nothing.

3) Is there a way one can build a module against a specific kernel source while running another kernel? I.e. running kernel "A" but want to compile against /usr/src/kernels/B/ ? If so, how do you do this? When I performed rpmbuild -bb SPECS/bcm5700.spec I was running kernel-smp-2.6.9-22.0.2.EL but the kernel source for that had been replaced with the source: kernel-smp-devel-2.6.9-34.EL by up2date and the rpmbuild failed. Basically compile/build drivers against multiple kernels without having to reboot into each specific kernel.

Wondering where you got this:
kernel-smp-devel-2.6.9-34.EL

Might want to remove that one based on my earlier words.

4) How many kernels should I leave installed on my machine? I am assuming it would be prudent to only have 2 or 3 versions (with their UP counterpart for a total of 4-6)??

I would leave the working kernel and a couple of backups in case a problem is discovered later. If you need 4 or 5 or 6, thats okay.

I think however, you've polluted your system trying to do something that was never necessary. To go from bcm5700 to tg3 and back, one merely needs to change /etc/modprobe.conf and reboot.

SEP

[Ryan Hobbs_2]

Shalom,

1) ..
No. Support for tg3 or bcm5700 is built into all of the above kernels. You should not have to do anything. I've found this out through real experience.

bcm5700 was not included in RHEL ES v4 and I had to compile that from source which I downloaded from HP's site. Currently, there is only one driver for bcm5700.

# find | grep bcm5700
./2.6.9-22.0.2.ELsmp/kernel/drivers/net/bcm5700.ko

So, when I boot to that 2.6.9-22.0.2.ELsmp kernel the module gets loaded. I am assuming it won't get loaded if I boot to the latest (or even non smp 22.0.2.EL) kernel. Therefore I have to compile it against every booted kernel I would use. Is this not correct? It is what I understood from the HP installation instructions.

2) ...
You don't need to do any compiles to get the bcm5700 driver to work

This is contrary to the information posted here:
ftp://ftp.compaq.com/pub/products/servers/supportsoftware/linux/bcm5700-8.2.14a-1.src.txt


The driver is released in a source RPM format. The file name for
the package is bcm5700-.src.rpm and is dependent on the
kernel source code.


3)..
> Wondering where you got this:
kernel-smp-devel-2.6.9-34.EL

I got that from RH. By running up2date -puf kernel-smp-devel.




> Might want to remove that one based on my earlier words.
I don't use clustering. Is something else broken?


4)
>I would leave the working kernel and a couple of backups in case a problem is discovered later. If you need 4 or 5 or 6, thats okay.

I think however, you've polluted your system trying to do something that was never necessary. To go from bcm5700 to tg3 and back, one merely needs to change /etc/modprobe.conf and reboot.

How is that? If I update modprobe.conf without the bcm5700 driver available for the kernel it renders my NIC's useless.
..again:
# find | grep bcm5700 (built manually by myself)
./2.6.9-22.0.2.ELsmp/kernel/drivers/net/bcm5700.ko

and for tg3 (supplied by RH)
# find | grep tg3
./2.6.9-34.EL/kernel/drivers/net/tg3.ko
./2.6.9-34.ELsmp/kernel/drivers/net/tg3.ko
./2.6.9-22.0.2.EL/kernel/drivers/net/tg3.ko
./2.6.9-22.0.2.ELsmp/kernel/drivers/net/tg3.ko
./2.6.9-22.EL/kernel/drivers/net/tg3.ko
./2.6.9-22.ELsmp/kernel/drivers/net/tg3.ko

I fail to see how you can modprobe a module in that is not compiled for a kernel....

[Manuel Wolfshant · Accepted solution]

Question 1: No, there is no problem. See also answer to question 4.

Question2: all automatic upgrade tools (apt, yum, up2date, yast) are usually configured no NEVER delete or replace the kernel (and related) packages, in order to avoid problems (what if the NEW kernel does not boot, but the old one was erased ?). However, the kernel-source is not a critical package, so usually the upgrade process will delete the old version after installing the new one. As far as I remember, there is an option in the config file of up2date which specifies which packages should not be deleted. Just add kernel-devel over there.
However I am not sure about how up2date works (I have given up using it after its' options drove me nuts) but there is always the manual way of doing things: for sure you can simply download the kernel-source (or -devel) rpm and manually install it using rpm (eventually with rpm -i --force; there is no harm in this case because the kernel-source tree is always installed in a separate tree and never touches anything else).

Question3:
With regard to Steven's idea to reboot after modifying modprobe.conf: no need to do that. Just unload the old drivers from memory (rmmod). In order to do that for a network card, one usually must "ifconfig down " that card.

As of the compile process for multiple kernel: usually NO ONE does that. Just stick to the kernel you need. If you really have to keep all those kernel versions, there are several methods available
- compile for one kernel at a time, keeping exactly (and only) that kernel-source tree. For UP and SMP the kernel source tree is the same, there is just a minor change in the .config file
- use a chroot system, for instance mach or mock (google for "mock fedora")
- make sure when you compile the driver that you specify EXACTLY the kernel source you want to be used. Most of the time the rpmbuild process takes all input only from the included spec file, which blindly looks at /usr/src/linux-2.6 (or something similar). In cases like that, you should create a symbolic link from the real directory to linux-2.6 (ln -s /usr/src/linux-2.6.9-22.0.2 /usr/src/linux-2.6). If you are lucky, the spec is smart enough to accept parameters from the command line, so you could use "rpmbuild -rebuild --define "param1=value1 param2=value2" package.src.rpm. However, more often then not this option is not available.
- The final and most tedious method is to edit the spec file included in the src.rpm and specify the full path to the kernel source. In the spec file there is a %configure section which should accept this parameter.

With regard to question 4: As of the number of kernels one can have.. there is no limit. But unless you are on a development machine which needs different kernels for different tests, there is no point in keeping more then the running one plus a backup or two.
Also, usually there is no need on keeping both UP and SMP versions of the kernel, unless you have conflicting applications some of which run on one kernel (say UP) while others require the other kernel (SMP). This happened to me with a VPN client - it does not operate correctly on SMP kernels, while the EDA tools we use are much happier on SMP kernels. At installation time, if the computer has a SMP processor, both kernels are provided because the installer simply does not know what the user will prefer to use.
I for never keep more then two kernel versions on the computer I administer (unless I am lazy after an upgrade)

And last but not least: one should never use rpmbuild as root. See http://dag.wieers.com/howto/bits/rpm-build-user.php and be creative. The reason behind: security. Never do anything as root unless you really have to. Compiling does NOT impose to be root.

[Ryan Hobbs_2]

Thank you for the replies. I see in /etc/sysconfig/rhn/up2date that there is a section where I can add kernel-devel & kernel-smp-devel to pkgsToInstallNotUpdate so that should take care of that.


Question3:
> With regard to Steven's idea to reboot after modifying modprobe.conf: no need to do that. Just unload the old drivers from memory (rmmod). In order to do that for a network card, one usually must "ifconfig down " that card.

That is what I do as well, however I don't usually reboot this server unless I have updated the kernel. Which is the main reason for this question.

> As of the compile process for multiple kernel: usually NO ONE does that. Just stick to the kernel you need.

When you compile or build modules not supplied by the distributor, when you update the kernel, it is my understanding that you would need to rebuild everything again (is this not correct?). How is it that "NO ONE" does this? Are they just lazy? or do they just choose not to upgrade their kernel? (any good reason not to upgrade the kernel? I thought they included bug fixes, etc.)

> If you really have to keep all those kernel versions, there are several methods available

I don't think I would really ever need but 2 (or 4 for smp setups). I just would like everything working for any kernel I *might* have to boot to. And therefore would need all 3rd party modules/drivers compiled for every available kernel required... and it would be nice to do that against different sources without having to reboot under each specific version installed in order for it to build. I checked out the .spec for the bcm5700 and didn't see any place to update the kernel source directory and there wasn't a %configure section. I did see a %build section with a 'make'. This leads me to believe there may be some parameter to pass make in order to use a specified kernel source. Any Ideas? I will have to look into the chroot stuff.. I'm not familiar with that at all. Is that how you run multiple kernel versions (SMP & UP) depending on the application that you are trying to use (in your VPN client issue) ?? I guess I was under the impression you could only run one kernel at a time.

> And last but not least: one should never use rpmbuild as root. See http://dag.wieers.com/howto/bits/rpm-build-user.php
I will definately look into that. Thanks!

[Manuel Wolfshant]

> it is my understanding that you would need to rebuild everything again (is this not correct?).

No, you do not need to do that. If the kernel is modular you just add the modules you need to the existing ones. The key is to use the same compiler (!) and to have the same "kernel version" (editing the first 5 lines of /usr/src/linux/Makefile usually takes care of that). Dig a bit on google, or start from https://www.redhat.com/archives/nahant-list/2006-March/msg00015.html and you'll find how to compile just the modules you need.

> I will have to look into the chroot stuff.. I'm not familiar with that at all. Is that how you run multiple kernel versions (SMP & UP) depending on the application that you are trying to use (in your VPN client issue)

In one givem machine you can only have one running kernel at a time. What can be done is to use virtual machines, for instance using vserver or xen. Or other methods.

[Ryan Hobbs_2]

>> it is my understanding that you would need to rebuild everything again (is this not correct?).

> No, you do not need to do that. If the kernel is modular you just add the modules you need to the existing ones. The key is to use the same compiler (!) and to have the same "kernel version" (editing the first 5 lines of /usr/src/linux/Makefile usually takes care of that). Dig a bit on google, or start from https://www.redhat.com/archives/nahant-list/2006-March/msg00015.html and you'll find how to compile just the modules you need.

I'm not talking about compiling all the modules.. just the ones I compiled myself.
How do you interpret this statment from HP?

"Kernel Source Code Setup
========================
The bcm5700 driver requires the presence of the kernel source code
and configuring the kernel source before building the bcm5700 driver.
The following steps need to be done once for each kernel that is
booted. For example, if the current kernel is UP (uni-processor)
and an SMP (symmetrical-multi-processor) kernel is booted, these
steps must be performed again to configure the kernel source for
SMP before building the bcm5700 driver for the SMP kernel."

I took that as meaning I can't compile the module while running A-smp against A-smp-sources and then copy the moudule.ko file

where
for every
copy it to /lib/modules//kernel/drivers/net

I took it as I had to build them against every installed on the system I wanted them to be availble if the kernel got loaded. Is this not correct?

I really don't understand why there isn't a simple answer to this. I think your link to the redhat list has given me some insight.. I am just too new to this that I just need some further clarification and perhaps too much hand-holding! :) I will definately give it a whirl with the -C option to make. I just thought perhaps there was something easy I was missing.. like not having to hand edit every .spec file for every module or software that I have to re-build for the updated kernel. i.e. KERNEL-SOURCE-PATH=/usr/src/kernel/Not_loaded_yet_kernel; export KERNEL-SOURCE-PATH; something like that.

[Manuel Wolfshant]

> I took it as I had to build them against every installed on the system I wanted them to be availble if the kernel got loaded. Is this not correct?

Yes. it is. And the simple way to do that is:
for every
ln -s /usr/src/linux-KERNEL_VERSION /usr/src/linux
cd /usr/src/linux
if SMP then
copy configs/KERNEL_VERSION-smp.config .config
else
copy configs/KERNEL_VERSION_PROCESSOR.config .config
endif
make oldconfig // end of configuring the kernel source
/*steps required to compile ONLY the driver come here, for instance
make -C /lib/modules/KERNEL VERSION/build SUBDIRS=`pwd` modules
the key is using the SUBDIRS option which requests compiling JUST the modules from the current working directory.. or whatever directory you specify
*/


Maybe http://ldp.azc.uam.mx/LDP/lkmpg/2.6/html/x209.html
will help you a bit too to understand the process.