"mv" commands runs too slow

[Yu Zhen]

I am running an application, which needs to move massive files from one directory to another. But the speed was too slow, about 2,000files/hour. These files are very small like some kilobytes. Any idea?

[Ravi_8]

what's the u r RAM and swap size, if less try increasing both.

[Yu Zhen]

ravi, I have got 6G memory and 10G swap space on a V2250 box, the storage is FC60.
I think the HW is sufficient for such a simple action,but the performance is really unbearable.

[Thierry Poels_1]

Hi,

are you really MOVING the files i.e. to another disk?
if you 'move' the files on the same disk they are simply renamed which is done a lot faster.
regards,
Thierry.

[Paula J Frazer-Campbell]

Hi

How are you doing the move with "mv", if so try a copy then remove.

Also run sar and look at CPU / Disk and Swap whilst it is going on.

Paula

[Yu Zhen]

Hi, Paula
I can't quite catch your idea. The CPU and swap usage are low, and why shall i make a copy and then remove them?

[Vincenzo Restuccia]

Check inodes frees with
#sar -v 5 5
and your maxusers kernel parameter.

[Klaus Crusius]

I observed similar performance degradations, when there were simply too many files (~20000) in source of traget directory.

[Paula J Frazer-Campbell]

Hi

A move command copies the file and removes the old so if doing a mv on many files :-
cp
rm
cp
rm etc for x times

whereas a copy of all first and go back and remove all=

cp all
rm all

far less jumping from dir to dir so therefire quicker.


Paula

[Deepak_5]

Hi,

I feel that mv is always faster than copy.
Only reason, mv would be slow if you are
moving files across filesystem.

Just an additional info, the inode number
changes only when a file is moved across file
system.
It remains unchanged, if it is within the same
filesystem.
One can verify by using ls -il command on that file.

mv uses rename() system call, if it fails, uses
copy routine(basicailly reads from a file and writes in another file).

This is what I feel, hope it helps.

Thanks and Regards
Deepak

[Darrell Allen]

May not apply here but a relatively quick method for copying files from one filesystem to another is:

cd /old_dir
tar cvf - . | (cd /new_dir; tar xvf -)

Darrell

[Bill Hassell]

The fastest way is to use the puddlemove options in cpio as in:

$ cd /source_dir
$ find . | cpio -pudlmv /destination_dir

cpio -p is optimized for disk-to-disk file transfers. Leave off the v option if you don't need the verbose (file by file) listing.

[Darrell Allen]

Thanks Bill. The cpio copy of a directory was considerably faster than my tar.

Darrell