topic strange - FS usage differs in Operating System - HP-UX

strange - FS usage differs

Prashant Zanwar_4 — Mon, 09 May 2005 13:35:00 GMT

Hi,
I am working on copying contents of a dir under a file system to a different file system. I am using rsync here, just to make to same. Below is what I am doing.

rsync -av /ora_dump/CTSPRD2 /ora_dump/NewCTSPRD2

I dont use compression because it's a local copy.

Problem I am finding is strange till now:

ohms# cd /ora_dump/CTSPRD2
ohms# du -sk *
1296 audit
1327 audit.old
712 babu
1629195 bdump
19711 cdump
1 connect_report
0 exports
1 ksmsp_report
166080 monitor
1253699 perf
161078 udump
==========================
ohms# cd ../NewCTSPRD2
ohms# du -sk *
ohms# du -sk *
2420 audit
2506 audit.old
712 babu
4908252 bdump
19710 cdump
0 connect_report
0 exports
0 ksmsp_report
0 lost+found
160624 monitor
1256534 perf
161148 udump
=============================
lvcreate -i 2 -I 64 -L 8192 -n vora_dump_ctsprd2 /dev/vgfs
==
newfs -F vxfs -o largefiles -b 2048 /dev/vgfs/rvora_dump_ctsprd2
====================================
I have tried chaning block factor to 8192/2048 etc. I can see the same usage difference,

ohms# bdf /ora_dump
Filesystem kbytes used avail %used Mounted on
/dev/vgfs/vora_dump
8192000 5445573 2574832 68% /ora_dump
ohms# bdf /ora_dump/NewCTSPRD2
Filesystem kbytes used avail %used Mounted on
/dev/vgfs/vora_dump_ctsprd2
8388608 6533104 1797578 78% /ora_dump/NewCTSPRD2
==============================================
I want to make it happen as ora_dump is common and I have seperate dir/s underneath it.
Anyone faced this or any suggestions..

Thanks and regards
Prashant

Re: strange - FS usage differs

RAC_1 — Mon, 09 May 2005 13:42:24 GMT

Files that are in use, will not be copied.
Are there any files that are in use??

lsof +D /dir1 - source dir.
fuser -cu /FS - source FS

Anil

Re: strange - FS usage differs

Prashant Zanwar_4 — Mon, 09 May 2005 14:14:15 GMT

Hmm..Destination is bigger in size.
Files are getting copied, but are getting double in size at destination.
if you see bdump size, it's quite big at destination.
I have copied when file system was down also..offcourse tried oevrwriting..didnt make a difference..
Files will be use as CTSPRD2 is live.
Thanks
Prashant

Re: strange - FS usage differs

Robert-Jan Goossens — Mon, 09 May 2005 14:25:54 GMT

Hi Prashant,

Try to eliminate to copy command.
Something like

# find /ora_dump/CTSPRD2 | cpio -pcmudv /ora_dump/NewCTSPRD2

Robert-Jan

Re: strange - FS usage differs

A. Clay Stephenson — Mon, 09 May 2005 14:38:11 GMT

This is probably the result of sparse files. When sparse files are copied the "missing" bytes are filled in with NUL's on the destination file.

You can see this for yourself by doing this:
mkdir /var/tmp/mydir /var/tmp/mydir2
cd /var/tmp/mydir
dd if=/dev/zero bs=1k oseek=1000000 count=1 of=x
cp x ../mydir2/

du -k /var/tmp/mydir
du -k /var/tmp/mydir2

Surprisingly, an ls -l of both files will show a 1MB (+1k) file size but the du will report very different results because the two commands are looking at very different things.

Re: strange - FS usage differs

Bill Hassell — Mon, 09 May 2005 15:43:18 GMT

As Clay mentioned, you will see a much larger result in the copy due to sparse files, a very normal Unix behavior. If a program crashes and creates a core file, this file almost always copies to a larger size (the extra space is filled with nuls). The file's behavior is identical, whether the undefined space is filled with zeros or not. Here's what happens: a serial read just advances to the next record and does not know anything about the missing records. The filesystem code will 'fill in' the missing areas with nuls. So the copy has real nuls and will be larger, but both files will comparer exactly the same.

It is also possible for the copied files and direrctories to be smaller. In this case, it is due to 'sparse directories'. When you create a directory (mkdir), it will be 2 blocks in size. Create thousdands of new files in the directory and the size of the directory (ll -d /some_directory) might be 512 blocks. Now purge all the files. The directory is still 512 blocks but contains nothing but 'empty' or reusable records. Directories are not truncated when files are removed, but if you copy the directory, it is created as 2 blocks and then enlarged when necessary.

The only thing you can rely on is the quantity of directories and files. Use find and wc to count for you.

Re: strange - FS usage differs

Prashant Zanwar_4 — Mon, 09 May 2005 15:53:30 GMT

So I guess while the source file system is not in use, shall it be ok to copy..
Thanks
Prashant

Re: strange - FS usage differs

Ermin Borovac — Mon, 09 May 2005 20:22:31 GMT

When source files are not in use, difference in size between source and destination should be smaller.

If size still does not match, there may be some sparse files, so you may want to run rsync with -S option.

-S, --sparse handle sparse files efficiently