Boot from SAN

[UxBoD]

Hi,

On behalf of a client I am looking at migrating their internal HP-UX 11.23 O/S disks to SAN disks on a EVA4000. I have tested cloning the root disks using DRD (Dynamic Root Disk) from internal to SAN, and that worked a treat. The cloned disks booted just fine.

Now my question is according to the HP Boot from SAN documentation this should only be performed on a disk group that is not high in application activity. Though was does this mean, as there is no criteria for measuring against. On the disk group at the moment is some applications, and Progress databases but most of these are pinned in memory through a caching mechanism.

Any ideas on this ? before I go back to the client and tell them they will need to purchase eight more disks.

TIA

[Steven E. Protter]

Shalom,

Tell the client its a bad idea.

Data on the SAN no problem.

Booting on the SAN, big problem. Because you can't boot the system when there is a SAN issue there is one less tool to diagnose the problem with. Actually many less tools.

One little mistake by the SAN admin and all eight systems are down.

Not all HP-UX systems support SAN boot and as for best practices, it is best to not.

SEP

[UxBoD]

Interesting though that the white paper (http://search.hp.com/redirect.html?type=REG&qt=HP-UX+boot+from+SAN&url=http%3A//docs.hp.com/en/7408/HPUXSANBootWP_final_v2_update.pdf%3Fjumpid%3Dreg_R1002_USEN&pos=3) only really lists the positives.

[Vladimir Fabecic]

I also had idea to boot HP-UX 11.23 from SAN.
I come from TRU64 and VMS background where boot from SAN is normal and sometimes you MUST boot from SAN (TRU Cluster).
Then I read post from SEP who said "Do NOT do it" and did some experiments.
As one from world where boot from SAN is preferred way to boot, I strongly agree with SEP!!!
Do NOT do it 11.23 or do it with 11.31.

[UxBoD]

Got a linky for SEP's post please ?

[Duncan Edmonstone]

SEP has very fixed ideas about boot from SAN (I expect born out of a bad experience). I would say it certainly has its place, but you need to have a reason for doing it. If you don't have a good justification, then don't bother. I think saying its not 'best practise' is too strong though, unless SEP can point to a HP document that clearly states that.

For the the real question is 'do I have control over/trust the SAN?'. If the answer is yes then I don't see SAN boot as anything particularly risky. However if my SAN is under the control of some loose cannon who is always making changes without telling me, I'd think very differently about it. In a a well managed/controlled environment boot from SAN is not an issue.

With respect to your original question, I'm not sure how we could arrive at any hard and fast rules - if you don't think the current load on the disk group in the EVA is particularly high then just create the boot vdisks out of that group, if you think you are already bottlenecked on IO, then consider adding more disks/another disk group for boot LUNs.

HTH

Duncan

[UxBoD]

Yes I have complete control over the SAN :) We shall still have the internal disks so periodically will clone the SAN boot image back using DRD. The reason why this is being looked at is they wish to upgrade a production server from 11.00 to 11.23. Has to be 11.23 as on PA-RISC. We have a loan machine where I have built up a copy of production, from scratch, on 11.23 and if I clone to SAN when the client wishes to perform the cutover I just need to down the upgrade server, present the LUN(s) to the production server and tell it to boot from them. The internal disks in the production server are very old aswell, and so is the whole N class. Having the O/S on SAN would certainly mean fast recovery to a DR server.

I do appreciate all comments that have been made, but it does feel to me, and from reading other threads that most issues have arisen due to poor SAN or HP-UX configuration; this includes firmware patching etc.

[Duncan Edmonstone]

>>when the client wishes to perform the cutover I just need to down the upgrade server, present the LUN(s) to the production server and tell it to boot from them.

Be careful...

This will only work without a hitch if your upgrade server is *completely identical* to your production server (i.e. same model, same disks in same slots, same cards in same slots).

If it's not then the chances are at the very least you will be paying a visit to LVM maintenance mode to export/import the root VG and other VGs, as well as having to figure out new instance numbering for your LAN cards. At worst you could find you can't boot due to missing drivers.

HTH

Duncan

[UxBoD]

the plan would be to mirror the configuration, but your warnings are understood :)

[Eric SAUBIGNAC]

Hi,

Several of my clients have their HP-UX boxes configured to boot on SAN. And there is no known issue due to boot on SAN.

Why do they boot on SAN ? Mainly for faster recovery.

For example some of them have 2 RX8640 and 2 EVA 4000, one of each in a separate room. vPar are configured inside the nPars : for one vPar "production" in an RX, you will find the same vPar "devlopment/integration" on the other RX, with the same IO slots. Each vDisk on one EVA is mirrored through LVM with the other EVA. So, have one room totally crashed, and you will be able to start production vPars in the other room, in place of corresponding integration vPar.

With HP Integrity VM, it's then more flexible if you boot the Guest's OS from the SAN : you can easly move VM from one HP VM Host to another, for availablity purpose or simply to have better load balance between VM Hosts.

Really, in my experience, there is NO problem with this kind of architecture, even less if you are the boss of the SAN, even less if you have redundant fabrics, even less if you do intelligent design ...

Just one fact : i do suggest you use SecurePath even if PV Links are supported with EVA 4000, mainly for performance reasons. Anyway, if you use securepath don't forget to add PV links in vg00 : at time where the system comes up remember that if the path used to build vg00 is down, securepath is not active. So you will not be able to boot.

Be fair with my english ;-)

Hope this will help

Eric

[UxBoD]

Eric,

Unfortunately the client will not invest in SecurePath, so I have rolled my own perl script which identifies from a SSSU capture config the primary and alternate paths :-

[upgrade:/]# evascan -s vgflxa1
Loading EVA Structure
Scanning Backplane for Fibre Cards
Identifying Fibre Device : /dev/td0
Scanning Fibre Interface for EVA Disks : /dev/td0
Identifying Volume Groups
Identifying Order Primary/Alternate Paths : vgflxa1

VDISK VG Name Disk Devices
------------------------------------------------------------------------------
037 vgflxa1 /dev/td0 [ /dev/dsk/c4t0d1 (PRI) /dev/dsk/c6t0d1 (AL1) ]
038 vgflxa1 /dev/td0 [ /dev/dsk/c4t0d2 (PRI) /dev/dsk/c6t0d2 (AL1) ]
051 vgflxa1 /dev/td0 [ /dev/dsk/c4t0d5 (PRI) /dev/dsk/c6t0d5 (AL1) ]
052 vgflxa1 /dev/td0 [ /dev/dsk/c4t0d6 (PRI) /dev/dsk/c6t0d6 (AL1) ]
054 vgflxa1 /dev/td0 [ /dev/dsk/c4t1d0 (AL1) /dev/dsk/c6t1d0 (PRI) ]
056 vgflxa1 /dev/td0 [ /dev/dsk/c4t1d1 (AL1) /dev/dsk/c6t1d1 (PRI) ]
059 vgflxa1 /dev/td0 [ /dev/dsk/c4t1d4 (AL1) /dev/dsk/c6t1d4 (PRI) ]
062 vgflxa1 /dev/td0 [ /dev/dsk/c4t1d7 (PRI) /dev/dsk/c6t1d7 (AL1) ]
064 vgflxa1 /dev/td0 [ /dev/dsk/c4t2d1 (PRI) /dev/dsk/c6t2d1 (AL1) ]

From this I have vgreduce/vgextend'd the volume group so that at least some balancing is performed across controllers.

Thanks to everyone for their input.

[TwoProc]

Ahhhh, one of my favorite subjects!!! YAY!!!

Re: SEP posting, the loose cannon SAN admin theory:

No, SEP didn't have troubles with a loose cannon SAN admin, he was his own SAN admin. And honestly, and without exagerration - you'd really be hard pressed to find better. If you're still booting from SANS then you've not had enough years of doing it to give it a chance to bite you in the you-know-what hard enough and long enough to have those days in hell that make you realize that it's just not worth it to make your life and all of those people that depend on you that darned miserable.

The point he made is that when you can't boot from the SAN, you can't even load tools to figure out what your problems are, you just don't come up. I too used to manage servers which were SAN booted and after plenty of "days in the ****" swore that if I could, I'd convert them to local SCSI drives as soon as possible, and I did.

When you're sitting there in a NON boot state, and EVERYTHING on the SAN looks fine from what terribly little you can see from a non-bootable server, and the SAN's switches all look fine from what you can see on them - you'll start making promises to God himself that you'll switch back, if ONLY he'd let you come up this once... :-) :-) :-)

Of course, are there valid reasons to booting from a SAN ? Yes, but not many, virtualization for alternate sites for disaster planning is one, but that can be done from local disks at both sites as well, but is more work, and thus booting from a SAN comes in as a real tempting solution.

But, from the same location? NAH.

There is a nice posting on this from the past:

https://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1007168

And, if you need help with that, a guy named Nelson Caparosso is IMHO one of the best at this whole thing of SAN booting and SAN admin.

BTW, he too got a bit critical of admins who don't like SAN booting, and insists that it's all so reliable, except for human error, knowledge and training (and I still respectfully disagree) - much like the point made above. So there's a support group "awdnkuwkoaos" for guys like me who STILL WON'T SAN BOOT THEIR SERVERS - join a local chapter near you today!

And, fortunately there are several 12 step programs
for guys who *DO BOOT FROM SANS* ...

...you know, to help get off the all the various types of chemical dependencies and help shake those mental disabilities that creep up on the admins over time from RMSASS (oft pronounced "rem sass")
"Repetitive Mental SAN Admin Stress Syndrome", etc. ...

:-) Just kidding! it's a joke! Oh Come On! :-)


Remember: Friends don't let friends boot from SANs

John
P.S. Gahd, did you ever notice that the guys in this forum can NEVER handle a joke? It's the worst I've ever seen in all of my days of being in forums or mailing lists - most people must be booting from SANs and have lost their sense of humor, it's kind of sad really... :-)

[UxBoD]

Well the machine still has its internal disks, so no harm from booting from SAN, but using DRD to clone to the internal ones as a backup.

[Duncan Edmonstone]

Oooh I can't let that one pass! - I'm not sure if we've ever had a holy war on the forums here - maybe its time we started one (that's a joke as well)!

>> No, SEP didn't have troubles with a loose cannon SAN admin, he was his own SAN admin. And honestly, and without exagerration - you'd really be hard pressed to find better. If you're still booting from SANS then you've not had enough years of doing it to give it a chance to bite you in the you-know-what hard enough and long enough to have those days in hell that make you realize that it's just not worth it to make your life and all of those people that depend on you that darned miserable.

Ok well if thats the case I won't argue - if it doesn't work for someone - don't do it - but don't assume that means it doesn't work for anyone. I don't think boot from SAN is some sort of panacea that we must all adopt forthwith, but neither would I classify it as 'not best practise' - and that was my point. I'm sure if we polled the forums we would find more people who would choose not to do boot-from-san than would choose to do it, but I'm also sure we'd find more people using LVM than we would VxVM - that doesn't make VxVM 'not best practise'.

>> When you're sitting there in a NON boot state, and EVERYTHING on the SAN looks fine from what terribly little you can see from a non-bootable server, and the SAN's switches all look fine from what you can see on them - you'll start making promises to God himself that you'll switch back, if ONLY he'd let you come up this once... :-) :-) :-)

Ahhh now if the only way we could get to a prompt was from boot disks we would be in trouble wouldn't we! Luckily HP gave us install media, Ignite servers and now DRD to provide other ways of getting to an OS prompt where we can diagnose SAN issues. The other usual counter argument I hear from most 'boot from SAN' proponents is 'so I can't boot my server, but even if I could, without the rest of its data what use is it?' That has some sense to it, but isn't really a strong argument (I hesitated even to mention it for fear of being lampooned!)

>> Of course, are there valid reasons to booting from a SAN ? Yes, but not many, virtualization for alternate sites for disaster planning is one, but that can be done from local disks at both sites as well, but is more work, and thus booting from a SAN comes in as a real tempting solution.

>> But, from the same location? NAH.

Yep, I agree completely - you need a good reason - DR is usually it, or system virtualisation (if you work on Superdomes with OS images in the order of 20+ it gets difficult, expensive and inflexible using local boot disks for everything)

So I think what I'm arguing for is a middle way between the 'wild horses won't make me boot from SAN' crowd and the 'why doesn't everyone adopt this wonderful technology' crowd, by giving reasonable advice:

1. Have a good reason for doing it.
2. Have a tested plan for diagnosing issues (how do I get to an OS prompt without the SAN?)
3. Make sure you have at the very least a veto on SAN changes as part of your change management process and ideally complete control over the SAN
4. Only use versions HBA/driver/firmware/switch/disk configurations supported by HP - if your switch/disk array isn't from HP then make sure you get the vendors agreeement that they will support boot from SAN issues

Cheers!

Duncan
(Counselor for SANBA - SAN Boot Anonymous)

[Eric SAUBIGNAC]

Hi Duncan,

many thanks for your answer : my english is too poor to do it by myself and in a quiet/moderate way as you did ;-)

Well, In fact I had begun an answer (with the help of http://babelfish.altavista.com/tr !!!) but I gave up when I saw yours. Mine was, how to say, a little beat less kind ...

PS : when i first decided to post this answer, HP forum was not available. May be it has been booted over a san ? (That's a joke !)

Eric

[Geoff Wild]

I have been booting from SAN now for over 6 months for our HP-UX vmguests. I havn't had any issues at all - works great!

We are about to receive some rx8640's and we are going to carve them up into 5 vpars each - with the first booting from the local disk, and the rest from the SAN.

Now, for a really cool real world application - say you lose a server - it totaly is destroyed - eletrical fire or whatever...

With boot from SAN, I just need to mask those disks to another server - and voila - I'm up and running....

Rgds...Geoff