Disaster/Recovery Test

[Richard Ross]

I will be testing a recovery of my 11.0 system at another site. Running a V2500 with an EMC Symmetrix. I have a question on the proper way to bebuild the VG's on the EMC. I would think once I recover VG00 from the make_recovery tape, I would then just need to issue a vgcfgrestore and newfs for each vg and fs. Am I missing anything? The big question is will my EMC be configured exactly as my production box. I'm expecting the device names (CxTxDx) to be the same, but I'm not taking that for granted. Even if the device names do not match, vgcfgrestore is still my answer if I use the -o option .. correct?

Thanks Richard

[Patrick Wallek]

When you restore your system on the V at the DR site, then only VG00 will be as it was on your production system. Your EMC will NOT be configured. The make_recovery tape will not create any LUNs on the EMC box for you. You will probably have to do that manually.

I think it would be much easier, once your system is up with VG00 recovered, to configure the EMC then set up your other VGs and LVs manually. Don't forget to verify that your /dev/vg?? directories and /dev/vg??/group files are set up appropriately as well.

[Richard Ross]

Patrick,

I guess I lost you ... When you say manual, do you mean starting with the pvcreate, mknod vgcreate ....? Can you explain further why the vgcfgrestore would not work?
The EMC should already be configured prior to my make_recovery.

Thanks

[Stefan Farrelly]

Because you cant be sure the EMC lun paths and sizes on the DR machine will be the same you really need some scripts to recreate all of VG01-X and their lvols before restoring the data onto them. This is what we did for our DR tests, even then we had problems because we asked for say 100GB and they gave us it but with unusual lun sizes which meant we couldnt have 1 VG of 60GB and one of 40GB, so write some flexible scripts and test them rigorously at the DR size, and ask for more space than you need, so you can be flexible with your vg/lv creations.

[Richard Ross]

Let me add one more point ... That EMC will be configured with our production bin file. The configuration should be the same, but I'm not sure on the device names ... EMC FE for that site said that they should match.

[CHRIS ANORUO]

It would be much easier, once your system is up with VG00 recovered, to configure the EMC since they servers don't have the same configuration and storage system

[Tracey]

I just recently had a disaster test on one of my 10.20 boxes, once I recovered from the make_recovery -A - the problem I had with the other volume groups is that it recovered their directories and lvmtab entries (hardware paths for original system). In order for me to manually create them, I had to remove the directories and remove/rebuild the lvmtab.

[James R. Ferguson]

Hi Richard:

I recently completed a D/R test at SunGard quite satisfactorily, recovering a K570 onto a K580. I used 'make_tape_recovery' for vg00 restoration. I would construct basic scripts that compose skeletal volume groups and logical volumes ahread of time. You will not be able to provide the actual disk device (/dev/dsk/xXtYdZ) files ahead of time since you probably will not know the actual hardware available until the time of your test, but you can go a long way to shortening and documenting the process by building scripts to create these entities. This shortens the time it takes to deal with dissimilar disk configurations that you will encounter.

Once volume groups and logical volumes are created and mounted, you can load your data accordingly.

It will take about 45-minutes just to load the golden image from the Ignite tape. All displayed messages will be written to /var/opt/ignite/local/install.log. Do *not* dispair if at the end you see a block-letter "FAILURE" message. The Ignite recovery will attempt to process the non-zero volume groups and fail due to differing/missing devices files in the vg map files it's using.

...JRF...

[Richard Ross]

Thanks for all the input .. Jim, what do your scripts do to shorten the process time? The main question, if the EMC is configured as production, shouldn't a VGCFGRESTORE and NEWFS work? This is a 2TB EMC with a lot of disks defined. Obviously, the time needed to restore the file structure is a factor.

[James R. Ferguson]

Hi Richard:

I think that it is unlikely that your disaster recovery (D/R) site will have the same physical paths (device files) that you have at home. In fact, depending upon contractual arrangements you have with your D/R provider, you might have quite a bit different disk pool and/or connectivity at the D/R site than you do at home. It is likely that you will begin your D/R recovery with only a pool of available disk from which it is expected that you pick-and-choose your configuration as you see fit. I don't think 'vgcfgrestore' is appropriate here. 'vgcfgrestore' is part of the process of restoring/replacing a single, defunct physical disk in an existing volume group, as you know. The filesystems spanning the affected disk must be rebuilt ('newfs') and restored from backup. In the D/R case you are building *all* your disks, volume groups, logical volumes, and filesystems from scratch. Scripting the 'pvcreate', 'vgcreate' 'lvcreate' 'newfs' processes provides flexibility and definition ahead of time. This brings me to my second point.

In my opinion, scripts are one of the best ways to define and document your environment. Obviously they're re-useable; and in a D/R situation I think they go a long way to speeding up the process of configuration and thereby shorten the time it takes before data can be begun to be reloaded. Remember, it may not be *you* that does the actual restoration. Ideally, you want a cookbook set of instructions that less experienced individuals can successfully follow.

Regards!

...JRF...