Has anyone successfully copied data from one LUN to another?

[Coolmar]

Hi,

I need to copy approx. 700GB of data from a LUN attached to serverA to a LUN on serverB. Here is what I have tried:

1. Restore from tape; scp; tar; all take way too long.

2. Add new LUN to VG of serverA; mirrored the LVs; removed the LUN from serverA and attached to serverB and tried a VGimport but no good because all lvm headers are removed during lvreduce and vgreduce from serverA.

3. Add new LUn to VG of serverA; mirrored the LVs again and then used lvsplit so that headers stay intact. Was able to vgimport on serverb and it all worked. HOwever, the LUN has to remain shared by both serverA and serverB. When removed from serverA (as we want the LUN exclusive to serverB) we lost everything on serverB as well.

4. pvmove will copy from LUN to LUN, but removes everything from serverA. We need the data on both servers, but exclusive to each other. If there was a pvcopy, then it would be perfect.

So if anyone has successfully copied data from one LUN to another LUN on another server...please let me know exactly how you did it.

Thanks everyone,
S.

[Steven E. Protter]

Shalom,

I've used the snapshot function of an EVA-4000 to copy data from one lun and create another. When licensed it happens in a few minutes.

EMC and other vendors have similar functions.

snapshot, change WWN and go.

SEP

[Coolmar]

Ours is a NetApp system, and we don't have that snapshot lisence...so we need to do it at the OS level if we can.

[Geoff Wild]

Yes, I have copied data between LUNS.

So, why not create a new VG on Server A, then copy the data (I use vxdump/vxrestore)

Example:

timex vxdump -0 -f - -s 1000000 -b 16 /APPL | (cd /zmnt/APPL ; vxrestore rf -) &

Where /zmnt is the new file systems.

Then, create a map file of the new vg, and vgchange -a n it on server a, vgexport it, then vgimport on server b.

Rgds...Geoff

[Sandman!]

Create a new VG (vgsvrB) on server B using a LUN that can be seen from both server A and server B. Note the device special files names for the new LUN on server A and on server B. Use dd(1) on server A to copy the data from the VG on server A (vgsvrA) onto vgsvrB i.e.

# dd if=/dev/dsk/c?t?d? of=/dev/dsk/c#t#d# bs=1024k

[A. Clay Stephenson]

The fastest transfer will be this one as suggested earlier (and he almost got it right):

Create a new VG (vgsvrB) on server B using a LUN that can be seen from both server A and server B. Note the device special files names for the new LUN on server A and on server B. Use dd(1) on server A to copy the data from the VG on server A (vgsvrA) onto vgsvrB i.e.

# dd if=/dev/dsk/c?t?d? of=/dev/dsk/c#t#d# bs=1024k
---------------------------------------------

Fix No 1: (Use raw character devices)
dd if=/dev/dsk/c?t?d? of=/dev/dsk/c#t#d# bs=1024k
should be:
dd if=/dev/rdsk/c?t?d? of=/dev/rdsk/c#t#d# bs=1024k

Fix Number 2:
You will be left with physical volumes which have identical volume group ID's. You should run vgchgid on the copied PV.

After that, a vgimport and you are in business. The dd method also assumes that the destination LUN is as large as the source LUN. The dd method also has the advantage that it will copy any raw volumes as well as fully cooked files.

[Jov]

Hi,

Doesn't NetApp Filers utilises NFS to share its storage? Is that not an option?


Jov

[Coolmar]

Thanks Clay and Sandman....I think I will try that. Just have a couple of questions:

1. Clay...not sure what you mean with "fix #2"...can you give me the exact steps?

2. Three of the logical volumes belong to Oracle. I assume Oracle has to be down when I do the "dd" but wanted to check. Also can the VG still be active when I do the dd or do you recommend it be inactive?

Thanks again fellows,
Sally

[A. Clay Stephenson]

1) No, I can't give you the EXACT steps becuase you didn't supply the EXACT physical volume paths (and I really hate questions like that because they suggest that you are too lazy to read the man pages for yourself). In any event, it's nothing more complicated than "vgchgid /dev/rdsk/cXtYdZ" specifying as many raw physical devices that comprise the copied volume group. A "man vgchgid" would have told you this.

2) It is all but essential that the filesystems be unmounted and is essential that any database be stopped. You are copying at a level delow the filesystem and even below the level of a volume group. Ideally, the volume group should be inactive as well so you should run "vgchange -a n /dev/vgxx" before doing the dd.

[Coolmar]

Well then the dd is no good as it cannot be tested prior to the actual server flip-over date. I cannot take those filesystems down and it obviously needs to be tested first and tested with those oracle filesystems.

There must be a way that we can do it with the mirroring....has anyone been successful with that method?

[A. Clay Stephenson]

While you may not be able to test with the exact volume groups there is no reason why this cannot be done with a pair of newly created LUN's --- one serving as the source VG and populated however you see fit and the other LUN serving as the destination. You can perfect your script on the test LUN's and then the only change will be changing the test LUN's to the real ones. This will also be a good method to determine the overall copy time as it should be directly proportional to the size of the LUN's.

If you have no raw data to copy then let me suggest a Plan 9:
1) Create your destination VG and LVOLS's and build the new filesystems on the destination server.
2) Shutdown your database and applications (briefly) and create OnlineJFS snapshot mounts of your database filesystems on your source server.
3) Restart your database and applications on your source server.
4) Copy using the utility of your choice
(probably an fbackup/frecover remsh pipeline) from the snapshot mountpoints on the source server to the corresponding mountpoints on your destination server -- or by using tape.
5) Umount the snapshot mountpoints on your source server.
6) Start your applications and database on your destination server.

You will have a exact copy of your database and applications data as it were at the moment of snapshoting; the beauty of this approach is that you don't care how long the transfer takes because your applications will be down only a few minutes. (A snapshot of a huge filesystem takes only seconds to complete because nothing is actually copied at that moment). If you script your shutdown, snapshot, restart sequences then total downtime might be well under 5 minutes -- depending upon the length of time required for database shutdown and restart.

[Bill Hassell]

> it cannot be tested prior to the actual server flip-over date.

Wow! This is a sure-fire recipe for a disaster. You don't have to test on live data but you absolutely must test using sample LUNs. Note that if the NetApp presents the files as an NFS mountpoint, then dd won't work as this is a physical copy, bypassing the filesystem completely. Note that pvmove is perhaps the very slowest method, followed by mirroring. The reason is that each extent is carefully read, then written then verified. Unlike dd, you cannot specify a large data block -- it's always done at the LVM level. In this case, mirroring means completely syncing the two volumes which for 700 Gb is going to take many hours. The good news is that mirroring can take place in the background and the data changing all the time without a problem. The snapshot method is likely the best solution.

[Coolmar]

Hi Bill...the LUNs are definitely seperate and speed is not an issue. I would prefer the mirroring - I know it is the slowest but it is the most reliable. I am still working on the lvsplit as there must be a way in there to do this. It all worked like a charm until I removed the disks from the VG on the serverA (the original server)...then it all blew up on serverB as they were "shared". So I need to find a way that after the split and the vg and lvs are on serverb that I can force them to be exclusive to serverb.

[Coolmar]

Finally figured out what to do:

Mirrored the lvs; created a map file using vgexport and copied it to the new server; then removed the LUN from the original server (do not lvreduce or vgreduce) and vgimport on the new server. Then lvreduce and vgreduce on the original server.

[Coolmar]

I tested this today and it worked perfectly. So here is what I did in case anyone else needs to transfer data to another server and both are attached to a SAN.

1. vgextend new LUN for new server (LUNB) into volume group on old server.

2. lvextend -m 1 /dev/vgxx/lvolx /dev/dsk/newlun
* do for each logical volume

3. # vgexport â s â p â m /tmp/vgxx.map /dev/vgxx

4. Copy the map file to the new server into /tmp/vgxx.map

5. Take the new LUN (LUNB) away from the old server. DO NOT lvreduce or vgreduce just make the disk unavailable to the old server at the SAN.

6. On the new server:
# mkdir /dev/vgxx
# vgimport -v -s -m /tmp/vgxx.map /dev/vgxx
# vgchange -a y â q n /dev/vgxx
# vgdisplay â v vg02
# mount â a (**after you add all the lvâ s to the /etc/fstab)

7. On the old server:
# lvreduce â m 0 /dev/vgxx/lvolx /dev/dsk/newlun
Then when all logical volumes are clear, do the following:

# vgreduce /dev/vgxx /dev/dsk/newlun

8. On the new server:
# lvdisplay â v /dev/vgxx/lvolx

You should see a row of ???
Now type:
# lvdisplay â v â k /dev/vgxx/lvolx

Where you used to see the ??? you should see a number instead â like 0 or 3 (something like that). This number is called the KeyValue.

# lvreduce â m 0 â k /dev/vgxx/lvolx KeyValue

After all logical volumes are reduced and the ??? are gone, do the following to remove the ??? from the volume group:

# vgreduce â f /dev/vgxx

# vgcfgbackup /dev/vgxx

If there are problems with the vgcfgbackup, then you still have traces of the old system disks. So carefully go through the vgdisplay and ensure there are no stale logical volumes. Then insure ALL logical volumes are mounted, and if not delete it or fix it. Then run vgreduce again. It will tell you to rename the /etc/lvmtab but you donâ t need to because you shouldnâ t have the xtra disks anyway. So just run vgcfgbackup and you should be back in business. If not, keep combing the vgdisplay.