Cloning/disaster recovery to unknown target system(s)

[Malcolm MacArthur]

We have a DR exercise coming up soon, and I have been doing some research in preparation for this. We contract out our DR to a company who provide us up with machines as necessary.

Previously, in the past, what we had done was to obtain the servers with base HP-UX 11.00 preloaded on them, then install our backup software (Legato Networker), then recover the backup software's index files, then begin recovering the systems. Ideally, it would be much nicer if we could restore a bootable tape with the Networker software on it, and just start recovering the systems.

Now, having come from the VMS world, this sort of thing is a breeze as long as it's the same processor type. Image backup your system disk, restore the image backup, reboot. I've looked into this on HP-UX though and it's a bit more complicated...

In VMS, the 'kernel' is tiny and is static (i.e. you don't get the objects or source to rebuild it). Consequently, the drivers are all outside the kernel, and a VMS system has to load the appropriate drivers into the system when it boots; this is done automatically (unless you turned it off). So naturally, DR is a breeze - restore the system disk image backup, reboot, fiddle a bit to overcome hardware incompatibilities, and off you go.

HP-UX, being Unix, does things differently (fancy that :)). In HP-UX, the kernel is monolithic, as it is in virtually all Unix systems, and any necessary device drivers need to be compiled in. By its nature, you would want to keep the kernel as small as possible because the whole kernel is loaded into memory and stays there. In order to do this, when you set up your system it only pulls the necessary drivers off of the CDROM and restores them; anything else is left off because it isn't necessary. The kernel is then built and the system reboots. All fine, until you make a tape and try and restore it on a nonidentical system...

So, I've been looking into how this could be done, and reading the posts here. I've established the following:

1. It is virtually impossible (read: theoretically possible but not worth the effort) to create a Golden Image on one architecture and restore it on another. That's OK though, I don't expect miracles from Ignite...

2. It is possible to create a tape on one system and restore it on another, but unless the source system has device drivers for the target system available for Ignite to do a mk_kernel and build the new kernel, then the effort is wasted.

3. You still have problems with things such as LAN interfaces, etc. I've been thinking about how this could be remedied; or at least made a lot easier - such as using /dev/cdrom as a symlink to reference your CD-ROM, and have a post-install script recreate the symlink.

Given all that, I have the following questions which I hoped some of you could help me with:

1. Is there a way (unsupported or otherwise) to copy *ALL* drivers (and to get the correct architecture ones!) off of the HP-UX install CD into the /usr/conf directories? I briefly looked into how I might do this using the Gigabit ethernet driver. However, I got stuck because there were 2 packages, GE-KRN and GE-KRN.2, and I couldn't immediately figure out which one was which... Could see where the files had to go, though. Would it be as simple as merging them into the /usr/conf hierarchy (once I had figured out which ones I actualy wanted), or would further work be needed? Basically I want to preload my system with *every* single device driver off of the install CD, so it can build a kernel for anything. But the usual install method would need a reboot, which is not a viable option... It would also place the unnecessary drivers in the kernel, and I'd have to go back and remove them...

2. Once you have all your drivers in place, ready to build a kernel, all I would have to do (famous last words...) is make an interactive recovery tape, boot off of it and reconfigure some stuff when the system reboots. This is my goal anyway. Is it possible?

3. Failing (2), I can do a make_sys_image of the server and create a bootable tape from this, with a depot on the tape containing ... well, I guess it would basically have to contain the core O/S (for drivers) and patches from the Support Plus CD-ROM. Problem is that with such an approach, it would take a longer time to reinstall, which may or may not be an issue - depending on how much longer it takes... Remember, all this is done with no Ignite server to hand.

4. What would you guys do? :)

Thanks,
-Malcolm.

[Geoff Wild]

First - if you are contracting out DR to a third party - then you are paying some good dollars - I would tell them to provide you with identical architecture to those systems you need to DR - I know the company I work for does that. Then you issue is easier - just use make_tape_recovery to restore OS/drivers.

If you have a lot of servers - and you have your own ignite server - then tell the DR company to provide you with a server for ignite - restore that system and catalogs first - then ignite restore the rest.

The company I'm contracted to does there own DR for critical systems - but are having a few non-critical systems "drop shipped" - we will be igniting those...

Rgds...Geoff

[Chris Vail]

I'd be awfully tempted to do this "The Unix Way", which means "tiny and stupid". Rather than Ignite or make_recovery, I'd use dd.

This assumes that the hardware is identical or very, very similar, and that the tape drive has capacity to hold the image of the entire boot disk.

Make your tape with "dd if=/dev/dsk/cXtXdX of=/dev/rmt/0". It'll take a while. Move it to your D/R system, boot it from CD, mount the boot disk to the CD, then reverse the dd process: "dd if=/dev/rmt/0 of=/dev/dsk/cXtXdX".

Its not elegant, but t'would work.

Chris

[Steven E. Protter]

I've just finished conducting a DR excercize which took place in HP's performance center. It was specifically designed so we could take our DR tapes to differenent hardware and see how hard it would be.

We did a cold database backup on fbackup and a make_tape_recovery tape of vg00, which includes /etc/fstab and all filesystems.

We specified that we needed in the DR center a backup device compatible with our Ultrium.

We booted the system off the tape, and restored VG00 and set up empty filesystems for the data. We had to answer two questions about destroying existing volume groups yes.

There was an extra boot when the target system compensated for the different network hardware that HP had and we did not.

The nice part is the make_tape_recovery included all of our patches and was not a golden image.

It is my conclusion that this methodology will allow you do restore your systems on similar but not exact hardware.

You can't go PA-RISC to Itanium under this scenario. You can however go rp5450 to rp5470, even rp5450 to rp2450.

That should be enough to get you through.

From start of excercize it took us 5 hours to bring up all production legacy systems. It took us the next three to verify it and erase the data.

SEP

[Sundar_7]

Hi Malcom,

I have been repeating this for quiet sometime now .:-) but you can use the following guidelines from HP to clone the systems

There are few guide lines to following before ignite-uxâ s make_recovery, make_tape_recovery, make_net_recovert and make_sys_image be used to clone the systems

1) First if the make_recovery or make_tape_recovery tapes are used to boot the systems, the source and the target systems should be of same architecture. This is because now ignite-UX uses different INSTALL kernels to boot depending on the architecture. The model of the system doesnâ t matter but the architecture of the system does.

The kernels are

WINSTALL - used for all the systems that are ONLY 64 bit (N, L , A etc)

INSTALL â used for systems that can run BOTH 32 and 64 bit HP-UX (K,D,R etc)

VINSTALL â used for V-Class systems

For example, the ignite-ux recovery tape created on a K-Class system cannot be used to boot a N-Class system since the kernel copied to the Ignite-UX tape is VINSTALL and the kernel WINSTALL which is used to boot N-Class systems is not available in the tape.

2) Second thing to make sure is that all the hardware enablement patches that are required in the target system should be installed in the source system. Say a recovery tape Is being created on a V2200 system to clone a V2600 system then all the hardware enablement patches required for V2600 should be installed in the V2200 system or else the system wil boot, create the kernel but when the system tries to boot from the kernel created itwill panic.

On other hand, if a V2600 ignite tape is used to clone a V2200 system then it will probably work becoz V2600 will have the newer patches which will allow the V2200 to create a bootable kern

[Malcolm MacArthur]

Ah yes I know that article Sundar, I've read it :)

How do I get my device drivers on the system without rebooting it? I don't actually want to install the devices; just have their object files available so that a new kernel can be built

Hardware enablement patches.... mmm, now that's a tricky one. But I take it they probably come as new object files, or as patches to existing ones, so...

[Malcolm MacArthur]

Ah, thanks Stephen. Yes, that's what's been tried in the past but no one can get it to work properly... However, that was using Ignite 2.something (I upgraded it this week), so maybe things will be better.

The thing is, I want something I can stick in the server and restore and it will 90-odd% work, to put things roughly. It's beginning to look like it's not possible, or nobody will tell me, cause it's a 'hack'. :(

5 hours is not bad; presently it takes about 8 or 9 to sort out our current systems to the point where we can begin restoring data. The major problem is that I can't justify making something and taking it there without knowing that it works. Or has a high probability of success. I could ask what systems we're getting and work towards that target, but that would make the exercise artificial, would it not? Because I wouldn't have that luxury in a real DR scenario.

This is something that should, in essence, be simple. Restore a tape and put the disks and filesystems in roughly the same places. Rebuild the kernel. That's all I want to do. The stumbling block is the second part...

How does a brand new installation build its kernel? Is there something from the OS install CD I could 'adopt' to do this?

[Malcolm MacArthur]

So.

I take it there is no way to do this and no plans to make it possible?