System replication and Lun Replication !!

[Anand Sreenivasan]

Folks,
I am in a fix and here is the situation, I have 6 HP systems in my production environment. I was asked to clone this in a remote site and so I took a ignite image and restored all vg00 on all these 6 servers in the remote site. The issue now is san based filesystem. We are using san based replication and I dont know what I need to do to get these replicated disks on my remote site. I tried VGSCAN but it didn't work. Do I need to export the mapfile from my prodcution environment to these servers and if yes, What are the steps I need to perform. Any insight is appreciated !! Thanks.

[Steven E. Protter]

Shalom,

You can dd right across the network using ssh and replicate LUNS if your storage provider doesn't provide utilities.

Check out these threads for details.

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1422017

http://serverfault.com/questions/51567/how-to-set-up-disk-cloning-with-dd-netcat-and-ssh-tunnel

SEP

[Anand Sreenivasan]

The LUN replication is already done in the SAN side and the luns are in the remote site now. So DD is kind of ruled out !! How do I import the data from these replicated disks given all data under /etc/lvmconf is intact. Thanks.

[DogBytes]

Have you broken replication, changed the mode of the copies to read/write and manually presented the disks to the servers?
Do you see the disks with ioscan?

[Anand Sreenivasan]

Yes, Replication is broken and I can see the disks !!

[DogBytes]

Then you should be able to copy mapfiles and vgimport. Example:

mkdir /dev/vg01
mknod /dev/vg01/group c64 0x010000
vgimport -m vg01.map /dev/vg01 /dev/dsk/...
vgchange -a y /dev/vg01

add to /etc/fstab and mount.

(you might need to fsck if source VG wasn't deactivated).

[Anand Sreenivasan]

As I said in my previous message, Vg01 file already exists on /dev in the target system since it was restored by ignite image. Can I directly do a vgimport in that case ?. Thanks again.

[DogBytes · Accepted solution]

If the group file already exists you can skip that step.

[Rita C Workman]

How was your data replicated between sites on the SAN arrays? EMC (SRDF) HP (CA)

Going from the idea that they are simply replicated to the other site

1. Create mapfiles at your primary site, so they would be included in vg00 (/etc/lmvconf.mapfiles) and now would be on the remote recovered servers.
2. Ensure that the arrays are split, to ensure that no communication/copying is happening between disks!!!
3. The disks at the remote site should be an exact match.
4. Provided the remote disk is zoned/mapped & masked to the proper server(s) then it should be able to see that disk.
......
If you had mirrored pairs on both sites; then the remote array disk is basically just a split off mirror. And you should be able to just start up the vg easily (provided the two sites have been split apart). When you go to activate the volume group, and set disks r/w, it will just look like your missing a couple disk, but the mirror copies should come up. Much like vg00 mirrored would reboot on the mirror if the primary disk went down. Same principle.

BUT, if the disks are not truly sync'd then that takes a bit more work I think. Since you'd have to identify each disk that was allocated and what vg it goes to and put the ctd for them in a file that you could use with your vgimport command to create the vg.

vgimport -v -m -f /dev/vg

Rita

[Bill Hassell]

There are some logistical questions that you'll need to address. If the disk-to-disk replication does not take a snapshot of the local disks before copying, you'll have to stop all activity on the local system until the data has been copied to the remote site. Otherwise, the busiest filesystems will be corrupted at the remote site as the data is constantly changing during the copy.

If a snapshot is performed by your SAN software, then the resultant copy (after replication is finished) will be a snapshot of a running system and all the filesystems will require fsck to cleanup the structures before mounting them.

AS far as your replicated VGs, on the remote side (before replication is started), you'll have to stop all applications using the replicated disks, unmount those filesystems and then deactivate and vgexport all those volume groups. Then wait for replication to finish. Once completed, use vgimport to bring back the volume groups, run fsck on the lvols with filesystems and then mount them and start your remote applications.

Note that there are some replication products that can maintain local snapshots at the remote site. In this case, you don't have to wait for current replication to finish. You can pick a particular snapshot that is already finished, perhaps from 2 days ago, stop the apps, export the volumes, run the commands needed to present the snapshot and then rebuild the VGs. This is very SAN-dependent -- some controllers will create different LUNs when a new snapshot is presented. This means that you'll need to have some sort of ID or serial number for each LUN that can be used to find the device file after the snapshot is enabled. Talk to your SAN administrator about how snapshots work.

Note that replicating vg00 using the SAN won't work as there are too many hardware-specific files that will be destroyed by replication and your remote systems will either crash or not reboot successfully if vg00 is copied from the local site. Keep the OS local only, and if necessary, move the application data to other VGs.

If your replication software cannot handle snapshots and you need some sort of realtime
replication (can be very expensive for networking), consider rsync to copy your files. rsync will skip files that haven't changed, copying only the changes. Because it is filesystem based, the updates can occur all the time, but your application will have to tolerate files changing by themselves. Databases will most likely crash under these circumstances since the changes are not going through the database engine and several depend on each other to be in sync while the DB is running.

[Steven E. Protter]

Shalom,

If you have SRDF, you can replicate LUN to LUN, even while applications are running. It is important that the logical volumes and volume group be exact copies of one and other.

The best way to do this without disk tools it to shut databases and applications and use rsync to copy the individual files across. This is fast and effective.

SEP

[Anand Sreenivasan]

Well, Thanks for all the suggestions and queries. This is what I ended up doing and seemed to work well. The replication is a thrid party software running on EMC san and we are quescing the data before breaking the link between original disks and replicated disks. I had to run vgscan on my remote site server and then ran a vgimport with the disk names. It got all the logical volumes and ran fsck since the superblock was dirty !! After the testing, I ran vgexport and deleted the device files using rmsf !! This procedure seems to work instead of doing a vgimport and vgexport with a map file !!! Any thoughts ?.

[Bill Hassell]

> I had to run vgscan on my remote site server and then ran a vgimport with the disk names.> It got all the logical volumes and ran fsck since the superblock was dirty !!

Correct. You are importing disks which are a picture of an active system. To avoid fsck problems, you would have to unmount the source machine's volumes, then perform the replication and finally vgimport the disks.

> After the testing, I ran vgexport and deleted the device files using rmsf !! This procedure seems to work instead of doing a vgimport and vgexport with a map file !!!

Removing the disk LUNs is fairly normal, but to bring back everything, you'll have to present the replicated LUNs on your disk array, then ioscan and insf the device files back again. For small systems with less than 50-100 LUNs, you can use vgimport to search for the LUNs as long as your mapfile was created with the -s option of vgexport. The VGID (serial number) of the VG is the first line in the mapfile and tells vgimport what to look for among all the LUNs.

If you have alternate paths, or use a path manager (DynaPath, PowerPath,, etc), you'll end up with alternate links in your VGs which may have to be manually removed.