Move tons of large files from SAN to SAN

[Muthyala]

Hi

Need to move large amount of files roughly 2.5TB od data from one LVM mount directory to another with in the same server. What is the fastest way of doing this.

TIA
Sri

[SoorajCleris]

Hi ,

I understand

/dev/vgxx/data mounted /data

other

/dev/vgxx/datanew mounted /datanew

you want to move /data to /datanew.


is /datanew on another storage?
if yes, is there any other LV on the VG?

Regards,
Sooraj

[Muthyala]

Sooraj

Your right /datanew is on another SAN storage. It thas only one LV for 2TB.

TIA
Sri

[Bill Hassell]

Is the data changing during the copy process? If so, this will require rsync to perform the copy. The first run may take as much as an hour and then subsequent runs will take just a few seconds. Note that only recent versions of rsync will handle large files (>2GB).

If this is a one time copy, an fbackup pipe will be the fastest but be sure you use a config file to maximize readers processes and record sizes. fbackup will handle unlimited file sizes. dd will also work if the filesystem is idle (no changes during the copy). Here is a good link:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1427976

[yunardi]

may be storage to storage replication can be use (i use true copy from HDS). I have regulary copy large amount of data in same server and also to another server within range of 100km, it takes about 30 minutes for first resync and less than 10 minutes for next resync (in daily copy). of course you will need to apy a license for this feature :)

For one copy only may be using cp (offline), fbackup and dd can be used, but definitely will take a long time depends on your server I/O, file type (size) etc.

[Raj D.]

Muthyala,

You can do with rsync (two time sync, first one all the data , 2nd one the changed data, with incremental rsync .

You can also use EMC open Migrator. If using EMC storage platform. You have to create another vg (same structure) , say vg_DATA_new , and once data transfered during cutoff-offline session, you can rename it back to original name.

Hth,
Raj.

[chris huys_4]

Hi Muthyala,

VxVM is offcourse the obvious solution. ;)

snapmirror/ snapshot the data from the original vxvm volume to the target vxvm volume and mount the snapshotted target volume to the new directory et voila, you have the fastest way to synchronize 2 vxvm volumes within the same vxvm diskgroup.

Offcourse, you will need first to convert the lvm volumegroup to a vxvm diskgroup, but thats pretty straightforward, and the vxvm snapshot functionality will require the "full" vxvm installed, i.e. a payable vxvm license is required.

Greetz,
Chris