RedHat says new SAN luns only visible after reboot.. you agree?

[Kevin_31]

Hello, I raised a call with RedHat and I'm hoping all your linux gurus working with SANs will be able to prove the guy wrong.

I'm testing putting RH ES (3.0 and 4.0) on DL360 G4 with Emulex Light Pulse Fibre cards.... and using it to run our Oracle databases on SAN disks.

These are the exact same servers my colleagues use to run Win2000 and Win2003.

Those servers can see SAN disks, and can see new SAN disks which are added without having to reboot the Win server.

Now, please don't tell me yucky old M$ Windoze can do this better than Linux!

The RH support guy said the luns are scanned by the BIOS and that Linux has no chance to see new luns until a reboot.

I've been trying the

echo "scsi add-single-device a b c d" > /proc/scsi/scsi

as well as unloading and reloading the lpfc module.

I've downloaded and installed scsidev-2.35 from http://www.garloff.de/kurt/linux/scsidev/ which seems to help getting the server to create the divice file (like /deev/sda, /dev/sdb) but so far I'm still having to reboot first.

My existing configuration is on HPUX, and there all we have to do is "insf -e" and "ioscan" shows the new lun with drivers ready to rock-n-roll. Please tell me Linux can do something similar!

points for anyone who can add to my limited knowledge on this subject

cheers,
Kevin

[Steven E. Protter]

My general experience is that new LUN's assigned to HP-UX can be picked up as you describe.

For some reason this functionality does not appear to be built into Red Hat.

I would suggest an experiment at this point.

SEP

[SAKET_5 · Accepted solution]

Hi Kevin,

We run a number of Red Hat ES3.0 Linux servers on Proliants (majority connected to the SAN consisting of EVA5000s and HSG80s). These servers have the Qlogic FCAs and there is absolutely NO NEED to reboot your Linux hosts to scan any new disk device as you present them to the host from your SAN. This is achieved as HP provides a "FibreChannelUtilities RPM" which contains a number of utilities such as "adapter_info", "hp_rescan", "hp_system_info", "lssd", "probe_luns" and "scsi_info". A README file ofcourse explains all these utilties and they work fairly well in my experience with scanning new disk devices etc online on Linux. The key message here is the tool I m referring to is for Qlogica FCA2214 and some other models of the card.

Check this out:
http://h18000.www1.hp.com/support/files/server/us/download/21485.html

Idea being if RH Support Guys are telling you that this is just not possible, then my above description has already proved them wrong..whether the OS natively has a tool to do it...I havent looked into that. Whether there is a similar utility as the one I mention before Emulex HBAs, whether the utility I mention will also work on Emulex even though it is probably not supported - I will let you figure out all these questions.

As far as your original question is concerned, I believe my post has answered it.

Good luck and regards,
Saket.

[Miguel Munoz]

Kevin,

As you can see from the responses that you have received the answer to your inquiry is 'it depends'. In my experience it dependes on the following things:
1. LINUX kernel you are running
2. HBAs you are using
3. SAN devices you are using.

Our shop has both HP-UX and LINUX servers (as well as Windows) connected to the SAN (Brocade switches and EMC Clariion CX600 disk arrays). On priniciple, I have recommended against any Database implementations on the LINUX platform (we use SLES9 from SuSE) until there is support for the exact functionality that you describe that equals the HP-UX platform.

Per our EMC documentation, the process to add LUNs without rebooting is similar to what you describe with the unloading/reloading of the module and that convoluted echo command. The recommendation being that there be no activity while this takes place (i.e. unmount the file systems).

Well... you may as well just reboot!

Adding LUNs w/o rebooting in this case is only a matter of semantics. Frankly I would as soon type "shutdown -r now" as to attempt to go through the error prone process described above.

As you can probably tell by now, I am an old school HP-UX bigot. That being said, I like LINUX a lot and am quickly being won over. I prefer to wait until the OS is a bit more stable before I recommend replacing my HP-UX DB servers with LINUX.

As a final note, the Windows functionality in the dynamic reconfiguration process ain't all it is cracked up to be. Just for fun have your Windows admins REMOVE some LUNs and see how long it takes for the servers to wig out. ;)

Good Luck,

Miguel Munoz

[Kevin_31]

Saket, can you comment on whether you unmount your filesystems before adding new luns--like Miguel says in the previous post of this thread.

I've got a call opened with Emulex and am hunting for a QLogic card to try it out.

I'll post up here what my tests show, but It'd be great to hear what others are doing already.

Thanks for all the info Miguel and Saket.

cheers,
Kevin

[SAKET_5]

Hi Kevin,

We do this on a day to day basis on Tru64, Windows and Solaris. I must admit we don't do it as often on Linux.

If I understand your problem correctly, you are adding new LUNs to the OS to create additional filesystems correct? I would be amazed if you had to unmount your filesystem (which is effectively taking your system down to single user mode ---effectively meaning service downtime!!) every time you have to add a LUN to the host from something like the SAN. Correct me if I am wrong, but the whole idea of storage virtualisation and SAN storage with the correct OS setup + multi pathing daemon + dual redundant HBAs + dual redundant fabric - is to perform as much stuff as possible online!

To my mind, you should not have to reboot your host everytime you wish to add a new LUN to it. Quite honestly, this would be less than pretty!

I would stand by my statement, that you should just use a utility to scan your new LUNs (with Qlogic FCAs, we have done it in the past with no problems).

I would wait for updates from you on response from Emulex.

Good luck.

[SAKET_5]

Kevin,

To add to my previous post, I would like to stress again that on a host running Tru64 UNIX, when we need to add a new LUN to the host, once the Vdisk creation + SSP tasks have been completed, all that is required on the host end is something like:
hwmgr -scan scsi

and then you view your new devices by:
hwmgr -view devices

The above probes the new LUNs and displays them without absolutely no downtime or without going to single user mode.

Similarly, for Solaris, after creation and presentation of a Vdisk to the host, you would run something like:
devfsadm -c disk
it probes any new disks for the OS.

For windows, you would simply perform a "Scan Disk" from within the device/disk management mmc.

SO, my point being if all these OSs can do it, I would be up for a shock if you tell me Linux cant - and that you would have to take the host down to get your new disks detected!! We already know this is not the case with Linux with Qlogic FCAs.

Hope, this helps.

Regards,
Saket.

[Bill Rothanburg]

Kevin,

We are using Emulex HBA's with our linux servers.

I don't remember having problems adding luns after the fact, but then I don't remember doing it either... :-)

Emulex has documented how to do this on Linux...

http://www.emulex.com/ts/docfc/linux/71/dynamic_luns.htm is for the latest revision of the driver.

The rest of the docs and drivers are at http://www.emulex.com/ts/docfc/linuxframe.htm


Bill