Data Migration?

[Dan Matlock_1]

I need to 'move/copy' large amounts of data from one SAN/LUN to a different SAN/LUN. I am trying to think of a fast/safe way to do this. I know the 'cp' cmd would be easy, but wonder if 'dd' or 'tar' would be faster..... Folks here don't trust the speed of our TSM (tivoli) backup to restore. I plan on slicing up the NEW LUN with one off name on the mount point, 'copy' then umount/remount w/proper name. We discussed LVM mirror, but app owner doesn't want to touch production VG (apparently tried this in past, and had bad results).

[Bill Hassell · Accepted solution]

The fastest disk to disk method is to use cpio -p. Just create a mountpoint for the destination, cd to the source directory and use the puddle-move options:

mkdir /destination
mount /dev/vgxx/lvolyy /destination
cd /sourcedir
find . | cpio -pudlmv /destination

NOTE: this is good for any size directory but limited to 2Gb files or less (cpio limit). Otherwise, and fbackup/frecover pipe would be faster than tar due to the multiple reader/writer processes and fbackup handles unlimited file sizes.

[Raj D.]

Hi Dan ,

Hope you got both the SAN/LUN filesystems mounted , so you can use

# cp -r -p /san1fs1/* /san2/fs1/

This is simple and reliable too.

Cheers,
Raj.

[Pedro Cirne]

Hi,

I migrate a storage last July, I used LVM mirror, it worked without any problems.

-It's safe.
-You can do it on-line, no downtime.

You have some contraints, depending on how the VG was created:

-You must add the new PV to the VG, and you have a limit for that, MAX PV

-If the VG is in shared mode you can't do it.

-If the lvol is LVM stripedd you can't do it.

Enjoy :)

Pedro

[Raj D.]

If you have lots of filesystem to migrate to new SAN , you can write a script and run at once , with monitoring string in middle of script so that you can come to know the activity as its doing.

We have transfered near about 200GB SAP data , from EMC DMX2000 to IBM DS800 storage recently with cp and script.

Cheers.
Raj.

[Mel Burslan]

from my personal experience last year, while migrating huge chunks of data from EMC to XP arrays, we used vxdump with quite good performance. The command we used was something like:

vxdump 0f - /${FS} | (cd /n${FS} ; vxrestore xf -)

we just prepended the filesystem name with a letter "n" to easily distinguish between the old and the new filesystems at the time of recounstructing the fstab.

Hope this helps

[Dan Matlock_1]

All info is excellent!!! I too in the past did copy from old srv to new srv using for loop type copy script along with cksum on old/new to make sure everything made it......

we are migrating/moving from a 2405 striped disk array (badly striped) to both xiotech SAN and IBM Shark SAN.....

[Raj D.]

This is what we did last time , for migration of data from EMC to IBM storage,

1. Create filesystems with new LUNS on New storage.
2. Mount the new filesystems
3. Bring down SAP and Oracle .
6. Copy Data from old file systems to new file systems. [ using cp -rpi , and script ]
7. Unmount old file systems
8. Move New filesystems to correct names.
9. Start SAP and Oracle
-------------------------------

may this helps..

Raj.

[TwoProc]

My favorite tool of choice for this is
cd /sourcedir
find . -xdev -depth | cpio -pdmvu /newdir

I like this as it restricts the gathering of data to the current mount point, and unlike cp maintains permissions AND ownerships correctly. "cp" also likes to create real files out of symbolic links, instead of recreating the links for you.

And per Mel's suggestion on vxdump, I've tried it, and he's right - it's really fast.

There's also tar for getting this done.
cd sourcedir
tar cvf - . | (cd /destdir; tar xvf - )

I like this because it keeps permissions and ownerships really well for you, and seems faster than cpio. But, mainly I like it b/c I like the display. What I don't like about it is that it will happily traverse mount points.

Picking my favorite of those, I almost always go for the find/cpio method b/c although I don't have nested mount points on mount points, I like being really confident that all I'm getting is what is on a single lvol. It always correctly restores permissions, ownerships, symbolic links, hard links, and even special files.

If you're going for throughput though - don't pass up giving Mel's advocated vxdump method a timed trial.

[Marvin Strong]

personally if would mirror if that was an option. its very easy.

outside of that, I have been doing tons of migrations this year, and been using the trusty

( cd $source_fs ; find . -xdev -print | cpio -pdumv $dest_fs )

I find it to be very fast.

[Dan Matlock_1]

tried the vxdump, but not sure if/how it works. Basically I need to migrate filesystem between SANs, and files will have: large files (oracle/sap DB), soft links, and of course requirement is to retain permissions! Whats the best approach?