Message to other newbies like myself about make_recovery

[Timothy Czarnik]

To all apprentice HP sysadmins:

If you are currently not creating make_recovery tapes of your systems, I would suggest putting a system in place to create these before all else. This is a free product contained in the Ignite-UX software at www.software.hp.com/products/IUX.

Last night, for reasons that I won't go into but every newbie sysadmin will face at one time or another (read SYSADMIN ERROR), I was forced to rebuild a K400 system. Using the make_recovery tape that I had created the night before (we do this nightly - overkill?), my system was back up and running in just over an hour with all data intact and all applications running (this also happened to be the OmniBack cell manager!). Without this, we would have been looking at a night full of OS/patch/application reloading fun!

Do yourself a favor and create the make_recovery tape!

-Tim

P.S. In the spirit of Halloween, more horror stories like the one above are encouraged to drive the point home!

[James R. Ferguson]

Tim:

Very good point. I think a nightly run of make_recovery is perhaps "overkill" as you stated.

You can always run (and parse the output of) 'check_recovery'. This utility compares the current system to the System Recovery
status file created by the last invocation of make_recovery. See the man page for 'check_recovery' more details.

...and Happy Halloween to you!!!...

...JRF...

[Patrick Wallek]

I wouldn't necessarily start relying on the check_recovery option in Ignite. I was at an HP presentation here in Dallas a few weeks ago and the HP reps there said that the '-C' option to make_recover and the check_recovery piece would probably be going away sometime in the next couple of releases.

They stated that because the -C option slows make_recovery down so much and that it has limited usefulness, at least as far as they are concerned, that it would probably be removed. By the way, I cut my make_recovery time in half by removing the '-C' from the command line.

As far as a nightly make_recovery.... Depending on what all is in vg00, that may be a good option. My backup indices are in vg00, so they get backed up as part of the make_recovery. I have done this so that when the system is recovered, all my backup stuff is there and ready to go.

I do a weekly make_recovery on my machine.

I would also caution you to set up a tape rotation schedule for your make_recovery tapes. You wouldn't want to be using only one tape and then discover when you need it most that you have a bad tape.

[Dave Wherry]

Continuing with Patrick's comment about having a tape rotation, Oh yessss!
I do a set of tapes every 3 weeks or after any significant config changes. My systems just do not change much. I use two sets of tapes.
I recently had LVM problems that I could not straighten out. I went back to my last make_recovery tape. The LVM problems were present there also. I went back to the previous tape. The problem was also there. It had been there for some time, but, because I had not made any LVM changes, I did not see it.
Luckily, in the back of my tape cabinet was a make_recovery tape from May. I loaded from that one and the problem was gone.
This relates to the point that your backups are not good unless you can restore from them. I had simply been backing up a bad LVM configuration on my make_recovery tapes.
I know this will be hard to do for most shops, but, if you can, test those make_recovery tapes by reloading a system.

Also, in the next couple of months I'll be working with make_net_recovery. Although the tapes are good, this is supposed to be much faster.

[Patrick Wallek]

I have recently started playing with make_net_recovery. It IS nice, however, HPs speed claim is a bit inaccurate. The speed of make_net_recovery depends a GREAT deal on several things: 1) Your network link between the ignite server and your client. 100 Mb would be a minimum. I don't think I'd try with 10 Mb. If you are lucky enough to have a Gb connection...... :) 2) The speed of both your client and your server systems will have a BIG impact. The file that make_net_recovery creates is gzipped, so depending on your data, this operation can take up quite a few CPU cycles.

You will want to make sure that you have PLENTY of disk space on your ignite server. I currently have a 23GB drive on mine, although if I were to back up many production machines this way, I would definitely want more space available.

Make sure that the largefiles option is enabled on whatever file system you are going to store your archives on. Also make sure you have the version of gzip that will support files larger than 2GB. The make_net_recovery will die when the file hits 2GB in size if you have the wrong version of gzip.

Here is my current set up and some of my experiences with make_net_recovery.

Ignite Server - C200 Workstation - HP-UX 11.0, 512MB RAM, 23 GB disk, 100 Mb network

Client1 - B132L Workstation - about 3 GB total data - 10 Mb?
Client2 - 9000/857S (damn near an antique) - about 9 GB total data - 10 Mb network

The make_net_recovery on Client1 takes about 2-3 hours to run. If I run a make_recovery on a local tape, it would take about 1 hour or so.

The make_net_recovery on Client2 takes...drum roll......17 Hours. Yes...17 Hours. When I run a local make_recovery it takes about 3 hours or so.

So, to summarize, make_net_recovery is a NICE tool....I don't quite believe HPs claim that it is so much faster than a make_recovery that is run on a local tape drive.

With the make_net_recovery, you don't have to worry about keeping track of tapes, but you do have to worry about backing up your ignite server. If you have some make_net_recovery archives that are larger than 2 GB then you can't do a make_recovery of them since tar doesn't support files larger than 2GB.

With make_net_recovery you have additional traffic on your network. Depending on when you run a make_net_recovery, this could also slow you down, depending on how saturated your network is.

Suffice it to say, right now I will stick with the regular make_recovery to a tape. I'll just use make_net_recovery as kind of a cool toy.