fastest way to copy files ??

[Stuart Abramson_2]

I'm copying files from one Oracle SID to another on the same system. Some 70 GB on EMC disks.

What is the fastest way to do this?

I normally do it one of two ways:

1. cd /dir
find . -depth -xdev -print | cpio -pdum /newdir

where: /newdir already exists!

2. cd /fromdir
tar cf - . | (cd /todir; tar xf - )

However, in this case I don't have any links or special files. I could use "cp".

So far I have copying using Method 1 above (find/cpio) and I'm getting about 1 GB/15 mins or 4 GB/hr, which will take me 17 hours. Too long.

Could I split it up into multiple streams and they won't interfere which each other?

[Chris Vail]

Yeah, you can split it up into multiple streams, and run 'em all in the background.
However, you're getting REALLY SLOW speed, and were I you, I'd look into why that is. On my EMC systems, I can copy 700GB in 4 hours using tar.

Chris

[Robert-Jan Goossens]

Hi,

One small suggestion is remove the -p option from cpio.

# find . -depth -xdev -print | cpio -cmudv /newdir

Hope it helps,

Robert-Jan.

[Shannon Petry]

cpio would be the fastest method of copy, as you have given the example for. Note that large files may/may not copy. Last I checked this was not supported by HP, same for dd. (I have never had problems, but that does not mean you wont).

Looking at your speed though, there is definately something wrong. I can copy GB/minute on a C3750 with ftp and rcp, so something is broken or... are these files in use?

If the files are not in use, then look hard at your network and make sure your settings are all as expected. lanadmin -x will give settings of the devices.

The only time I see performance this bad, is on 100baseT full duplex networks where the HP keeps switching or is always running at 1/2 duplex.

If it's duplexing, then
lanadmin -S 100FD
will change the settings to full duplex. Make this permanent by editing /etc/rc.config.d/hpbase100conf

If the lan interface is okay, maybe a bad port on the switch.

Regards,
Shannon

[Mark Greene_1]

Go with cp:

FILELIST=`find . -depth -xdev -print`

for FILENAME in `echo $FILELIST`; do

cp $FILENAME /newdir/$FILENAME

done

If you do not have multiple HBA's on your system, then no matter what method you use in Unix, you are dragging the data over the fibre and through the single HBA twice, which is what well may be constricting your bandwidth. Splitting the copy process into streams is going to compound the issue.

To be thorough, you can check the fibre controller for errors. Look in the syslog to. You may have an iffy cable or HBA.

HTH
mark

[Stuart Abramson_2]

1. Why will cpi "-p" slow down my copy?

2. tar is SLOWER than cpio. I'm getting 6 GB/hr with both tar and cp.

3. I'm on the SAME system. there is no LAN involved.

4. 700 GB/4 hrs = 175 GB/hr. That's pretty fast.

5. It takes 2 hours to reload this SID from tape. 70 GB/2 hours = 35 GB/hr tape to disk.

[Stuart Abramson_2]

I have multiple HBAs and EMC PowerPath.

[Stuart Abramson_2]

Well, I have calmed down a little, and here is where I stand:

From 3:30 - 4:00 PM:
cpio: 4 GB/hr
cp: 8 GB/hr
tar: 6 GB/hr

At 4:00 PM:
two copies at same time:
cp: 60 GB/hr

So, I suspect that something was going on on my system, and I didn't know it, and it must have just stopped. So all my previous timings were completely "invalid", and I will proceed and just hope to finish in time.

Thanks for all your help.

[Yogeeraj_1]

hi,

i had a similar question a few months ago and got some great replies from the gurus here.

see

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xb20b5bd3782dd711abdc0090277a778c,00.html

hope this helps!

best regards
Yogeeraj

[Robert-Jan Goossens]

Hi Stuard,

I made a big mistake, sorry sorry.

The command you used for cpio is the fastest. I mixed the verbose and pass-through options.

Hope you accept my apologies,

Kind regards,

Robert-Jan.
0 point please for both answers.