disks seen in different order on two netservers?

[Lucy Berlin]

I have two LC-2000s with 2 internal disks each. Both servers are conneced to two storage units (A and B). Usually fdisk -l finds the same shared disks in the same order, but yesterday one netserver assigned device file names in opposite order (B's disks, and then A's disks). I rebooted and got the same behavior: /dev/sdc didn't refer to the same disk.

What could be going on?
And, how do I keep that from happening again?

[Kodjo Agbenu]

Hello,

What distro are you using ?
Did you recently compiled a new kernel ?
Do you have exactly the same /etc/lilo.conf on both servers ?
Did you check BIOS settings ?

Did you activated the following kernel compile feature ?

-> CONFIG_BLK_DEV_OFFBOARD (in kernel configuration file /usr/src/linux/.config) ?

This parameters allows the PCI devices order to be re-defined.

Did you change the SCSI IDs of the disks ?


I hope one of these helps.

Good luck

Kodjo

[Albert P Tobey]

Most of the time when I see problems like this, it's the network interfaces that are getting messed up. This is usually because the drivers are loaded as modules and end up getting loaded in the wrong order. Try recompiling your kernel with your scsi drivers compiled in and see if things are a bit more consistent.

Another solution I sometimes use is to modprobe the drivers in /etc/rc.d/rc.sysinit toward the top so that autoloading doesn't get a chance to screw up the order. This is not usually a good idea, though, as system upgrades can overwrite your changes.

[Eric Ladner]

Another thing to check.. Did you recently change the order of the PCI cards in one of the systems, or change the cables connecting the systems to their external devices?

Order matters on a linux/scsi system if you are not using devfs.

Eric

[Lucy Berlin]

Both nodes use the same driver for both storage units, so I don't think it could be a driver loading order. It appeared that after a while one of the nodes was not able to see the disks on one of the array's at boot time (but could after the boot).

After re-seating the scsi cable at both ends (with no changes to cabling, lilo.conf ...) the problem went away. Now the nodes see all devices in the correct order -- even after reboots.

So the problem's gone for now, but I still would like to understand what happened.