Checklist Before Reboot/Bootdisk Change

[joe_91]

Team:

I feel this is very important. Numerous times we have faced this issue where when we change the primary boot disk(bad disk) we have issues with the machine not coming up after replacement(sometimes the reason may be the wrong procedure and some times the reboot itself is an issue 'coz of changed files/dir). Can we come up with a list of things to check to make the machine will come up after reboot and also some particular considerations for boot disk replacement.

Thanks

Joe

[James R. Ferguson]

Hi Joe:

HP has provided an excellent guide to disk replacement scenarios -- boot disks and non-boot disks; mirrored disks and non-mirrored ones; hot-swappable and non.

Pay particular attention to the 11i LVM OLR (Online Replacement pathces) discussed in the guide. If you don't have them installed, you should plan a time to reboot to install them *before* your next problem.

The guide exists here:

http://www.docs.hp.com/en/5991-1236/When_Good_Disks_Go_Bad.pdf

Regards!

...JRF...

[joe_91]

JRF:

Thanks. what i was looking for was a general set of commands, files, directories etc to check for before a standard reboot or disk replacement checklist BEFORE you actually start the reboot or disk replacement.

Thanks

Joe

[James R. Ferguson]

Hi (agin) Joe:

I'm not sure for what you are looking. You should have the EMS hardware monitoring tools installed to alert you of problems and failures. EMS tools are a part of the Online Support Tools which you should keep up-to-date along with your patches.

If you have to replace a disk, the sooner you do so the better. Follow the procedures in the guide I pointed to and you should be fine.

As for routine, planned reboots (for patch application, for instance), I'd look for any events in '/var/adm/syslog/syslog.log' of concern before your reboot. Again, though, with proper monitoring tools in place, you should already have had notice of any problems.

Regards!

...JRF...

[A. Clay Stephenson]

If you stop and think about this for a moment, you will realize that there is really no way to insure that a machine will boot after a disk replacement. There are simply too many things that have to be just right and only one thing that needs to go wrong. Moreover, your reason for the machine failing may have absolutely nothing to do with a failed boot and and replacing it does no good whatsoever. If you follow the procedure outlined in the document that James mentioned then you should have no problem.

The most important thing that you can do is mirror your drives so that the failure of a drive is a complete non-event. Your users won't even know anything has happened. I have replaced hundreds of drives (including tens of boot drives) and have never had a hiccup -- but I follow a strict replacement procedure exactly, each and every time.

Now there is a problem that mirrored drives won't fix because the mirror obediently copies your mistakes such as really bad patches or your own stupdity (e.g. rm -r *). For those kinds of problems, I always have "lifeboat" disks on my machines. A lifeboat is dd'ed raw copy of your running boot disk(s). Each week (or before patching), I create a fresh lifeboat copy and if anything goes horribly wrong, all I have to do is yank the original boot disk(s) out of their slot(s) and insert the lifeboat disk(s) in their place and I'm back in business in a fraction of the time an Ignite would take. Oh, and because I'm paranoid, I also make Ignite images.

[Bob Ingersoll]

If you are asking how to check your system to ensure that it's properly configured to boot from an alternate disk then you may find these commands usefull:

Use lvlnboot to verify that your primary and alternate disks are both bootable:

lvlnboot -v vg00

Use setboot to ensure that the pri and alt boot paths match the disk paths shown by lvlnboot:

setboot

Use lifcp to verify the LIF on each disk contains the appropriate bootstrap:

lifcp /dev/dsk/cXtYdZ:AUTO -

The bootstrap should contain:
hpux -lq

And, finally you should test by shutting down, removing the primary boot disk and booting. This will verfify that the alternate boot disk can be booted and that autoboot is configured properly.

[A. Clay Stephenson]

Actually, even booting from the alternate boot disk doesn't guarantee that your box will be able to boot next time. While getting the scheduled downtime to attempt a reboot after a disk rebuild is a good idea, it's still only an indication that the machine was able to boot at that time. For example, let's say that you did successfully boot from both your primary and alternate boot disks (and that is certainly good news) but that was last week and 30 seconds ago you just removed (accidentally and unknowingly) /stand/vmunix* or any other critical files you choose that will not be needed until boot time.

The best you can really do is be disciplined about your disk replacement procedures and your system management procedures. As long as you are disciplined about your disk replacement procedures and follow a strict checklist, disk replacement itself is almost a non-issue.

[Steven E. Protter]

Shalom Joe,

Nothing in life is certain. You can look at the document, the stuff I'm going to show you about mirroring a system below and still, something can go wrong.

The best insurance policy you can have is a make_tape_recvoery backup handy from a working system.

Here is my mirroring guide.

pvcreate -B /dev/rdsk/c1t0d0 #use real disk

mkboot -l /dev/rdsk/c1t0d0
mkboot -a "hpux -lq (;0)/stand/vmunix" /dev/rdsk/c1t0d0 # use real disk


# mkboot -b /usr/sbin/diag/lif/updatediaglif -p ISL -p AUTO -p HPUX -p PAD -p LABEL /dev/rdsk/c?t?d?

If you are running 64-bit OS:

# mkboot -b /usr/sbin/diag/lif/updatediaglif2 -p ISL -p AUTO -p HPUX -p PAD -p LABEL /dev/rdsk/c?t?d?


vgextend /dev/vg00 /dev/dsk/c1t0d0 # same thing
lvextend -m 1 /dev/vg00/lvol1 /dev/dsk/c1t0d0

# real disk. repeat for other lvols

lvlnboot -r /dev/vg00/lvol3 # root fs /
lvlnboot -s /dev/vg00/lvol2 #swap
lvlnboot -d /dev/vg00/lvol2 #swap/dump
lvlnboot -b /dev/vg00/lvol1
lvlnboot -R
lvlnboot -v
setboot
setboot -a 52.1.0 # second disk

Thanks to all that made this doc right.

SEP