Fastest copy method

[Don Spare]

We are planning a migration from EMC Symmetrix to Clariion. As the new LUN sizes will be larger than on the Sym it seems the method of data migration will be to create new filesystems on the Clariion and then 'cp' the files from the old filesystems to the new. There is about 80Gb of data to transfer 99% of which is Oracle data files. Both the Sym and the Clariion will be direct connected over fibre channel.

My question: Is this the fastest way to copy the data? Is there another 'copy' command that works faster?

Thanks for help.

[Alzhy]

safest would be:

vxdump 0f - /emc_mnt|(cd /clarion;vxrestore rf -)

[Mark Grant]

We did a similar thing on Saturday copying 650GB of data from an XP to EVA and most of it was oracle stuff too. We did 650Gb in about seven hours using "cp -rp . /newlocation". I would guess that assuming a 2GB SAN, it will copy 80GB moew quickly than you can test and implement a faster copy :)

[Alzhy]

Or if uptime and availability is most important, use mirroring on the LVM side... Add the clarion LUN, mirror the LVOLS, once synched up, drop the EMC...

[A. Clay Stephenson]

Here is one "sneaky" method:

1) Create a new LVOL on the new array (as big as you like so long as it is at least as large as the LVOL on the old array).

2) Umount filesystem on old array.

3) dd if=/dev/vg02/rlvol1 of=/dev/vg05/rlvol1
bs=1000k

4) Change /etc/fstab so that the mountpoint is changed to reflect the new VG.

5) Mount the filesystem on new array.

where vg02 is the "old" VG and vg05 is the "new". You probably need to play with bs to get optimum transfer speed. Because we are raw copying the underlying filesystem data structures come over exactly as they were. You can also use fsadm -b after the dd to "grow" the filesystem to fit the larger LVOL so that you can set up the filesystem to be as big as you like.


Obviously, you could have as many of these dd's goinmg as you like.

One "gotcha" with cp is preservation of owner/group and permissions.

[Alzhy]

I finf 64k block size when using the DD approach to be th emost optimal for most SAN LUN's I've worked with.

Another disadvantage to using "cp -rp" is it does not cleanly copy "special" files.

So there you go:

1. vxdump / vxrestore
2. dd method
3. cpio

all of these preserve ownerships and file integrity...

[Geoff Wild]

vxdump is the fastest - migrated last year from 8730 to DMX1000's.

vxdump -0 -f - -s 1000000 -b 16 /data/work | (cd /data/worknew ; vxrestore rf -)

Rgds...Geoff

[Mark Grant]

The only problem with all the above solutions above is that you tend not to end up with a system looking exactly the same as it did. Your logical volume names change for example.

I agree with A. Clay about being careful with the "cp" command which is why I would suggest you "cd" to the filesystem you want to copy and then do "cp -rp . /newlocation".

We decided renaming volume groups was the easiest thing to do so created new volume groups on the EVA with identical logical volumes to the ones we were copying. We created new filesystems and mounted them to tempory locations. We then did the "cp .rp .". When we were happy, we remounted the new filesystems on the old mountpoints. When we were happy everything was OK, we exported the old XP volumes groups, exported the new ones and imported them again with the old names.

This way we could be sure that the system looked exactly the same and any script anywhere on the system doing things that it shouldn't be, i.e depending on the filesystem device name would still work.

[Alzhy]

Mark, the method we are suggesting is a "Generally Safe Method" for migrating filesystems. Try doing a cp -rp with a filesystems that has sockets, pipes and other special device files on it.

Besides, if the intent is to have the migrated filesystems look exactly like the old (meaning it's LVOLS as well are named the same).. the fix is easy.. export with the map file option.. edit the map file and re-import your migrated VG with the orginal names.

We should be careful here IMHO.. I've seen a number of Junior admins out there that followed a track that "..hey this worked! so this should work for all cases...". And I've been burned once trusting that one solution fits all...

[Mark Grant]

Actually, the above solution was suggested to us by an extremely senior (and extremely expensive) storage guy from HP Norway.

I must admit, the mirroring option seems attractive to me, especially as we have to export and re-import our volume groups anyway. I might try this for our next move next week.

[Alzhy]

And as far is which is the fastest:

dd would be the fastest
vxdump just a tad slower
cpio would come last

and forget "cp -rp" from now on. If it's not whole filesystems that you are copying/migrating -- the dd and vxdump/vxrestore* is out of the question... your friend here is "cpio".


(Although VxFs 3.5 I think now supports vxdump/vxrestore of directories on VxFS filesystems....)

[A. Clay Stephenson]

The mirroring model breaks dowwn (almost certainly) in one crucial area. You said that you were going to larger LUN's. By default (and unless you specifically overrode this when you set up the VG), vgcreate sizes max PE's to the size of the largest physical volume (LUN) listed on the command line. Once set, this value cannot be changed. Any LUN's larger than this will have space that is not used -- ever.

That is why I did not suggest it in the first place.

[Alzhy]

Good catch A. Clay!

But is there even an iota of possibility that he's using VxVM instead of LVM? For which LUN sizes will not matter much if using the mirroring approach..

Or if they are indeed using LVM.. possible that the good Admin has set up VGs that was "forward looking".. meaning expecting the biggest of LUNs.. We do.

[Todd McDaniel_1]

I was thinking of BCVs from EMC to the Clarion... I dont think BCVs are limited by the need to match PEs exactly... so long as you are going to a larger LUN for the copy. Although it is not an ideal long term setup, it should work here.

Do you have the capability to map BCVs from one frame to the other? Are they both assigned to the same Server?

The only flaw to my plan would be if the BCV compatibility is not good...

[Alzhy]

Very good Todd!

An even elegant solution...

I keep on forgetting Clarion is now part of EMC.. Last I've heard BCV's can extend to Clarions as well..

[Todd McDaniel_1]

Thanks...

Every once in a while even a blind squirrel can find a nut!!!

[Don Spare]

Thanks to everyone for their input. This gives me much to consider.

As far as the BCV issue is concerned, what EMC is telling us is that the LUNs have to be the same size for a SANcopy to work. Mapping the BCV from the Symmetrix to the Clariion was never discussed and if it can be done might be a solution for at least one server's worth of data (we only have one using BCVs). The rest of the data will have to be copied as I described in my original post (unless there is some other magical method that EMC hasn't talked about yet.)

[Todd McDaniel_1]

If the PHYSICAL disks are compatible, you can rehost them on the Clarion, then do the BCV snapshot.

Of course that is a lot of legwork and can be tricky.

[Hein van den Heuvel]

Just do it. Don't even worry. Just 80 GB? Running at say 80MB/sec? That'll be 1000 seconds. Is that a problem? If it is please explain as that knowledge will help define a better solution.

Mind you, I would encourage to basic base performance test, possibly using the real data, but perhaps better use /dev/null to write to and /dev/zero to read from. Just make sure you can run at the expecteed speeds, notably on the new device. You want to know for a fact that it is performing as expected before putting it into production no? So be sure not to find out during the real, can not go back in time, copy but find out now while you can still tweak and/or fix.

Hein.

[Geoff Wild]

With multiple paths, and setting up multiple vxdump streams, I migrated 1 TB in just over 4 hours. Our DMX was Striped/mirrored across 112 x 146 GB disks. If your Clarion is setup using "parity" RAID, then your time will be slower - as every write requires a parity re-calculation - with only 80GB - you shouyld still be able to migrate in less then an hour...

Rgds...Geoff

[MANOJ SRIVASTAVA]

Don

dd with a 1K blocksize , and then dont forget to do the extendfs so that the newer size is reflected.


Manoj

[Hein van den Heuvel]

Manoj, please explain why you believe in "dd with a 1K blocksize , and then dont forget to do the extendfs so that the newer size is reflected."

That's abous as slow as you can possibly make it (512b being the slowest :-).
Now if you had said ... 1024k
If dd is the weapon of your choice, IMHO you should use at least 64K blocks, but preferably more in order to hit a good bandwith and not be bogged dwn on exsessive IO/sec rates.

JMHO,
Hein.

[A. Clay Stephenson]

You definitely don't want a 1K dd blocksize. That is certain to be a dog. The minimum value I would consider is 64k; the maximum is probably around 8192K; 256K is probably a good all-around value. (The differences will not be all that great between 64k and 8192k because the OS itself will break large requests into smaller chunks; 1024K off the top of my head) but small chunks will definitely be a killer and 1K is small.

The other thing if using dd for this task is to NOT USE the seemingly equivalent ibs=64k obs=64k but rather use bs=64k. When using ibs= and obs= there is a copy operation from the input buffer to the output buffer that must be done whereas when using bs= there is but one buffer for both operations.

[Tim D Fulford]

Hi on the block size issue..

I used dd to migrate LVs from FC60 to VA7410 recently. I did some tests & saw that as you block size doubled the migration time halved up to about 512K or 1MB.. I decided to use 4MB as that is the extent size and got a pritty good transfer rates.

The only bit of advice is enable pre-fetch (if it exists on your storage device)

Regards

Tim