Incorrect file size reported

[user001]

Hello,
I have the following disk layout
vg00
lvol1
/mnt1
vg01
lvol2
/mnt2

I'm trying to copy all the data from /mnt1 to /mnt2, i've noticed when copying an 800mb tar archive from /mnt1 it reports the file is 2GB on /mnt2. All files copied from /mnt1 to /mnt2 seem to have this problem.

11.31
vxfs v6
20GB lv's

Any ideas. This is driving me insane, /mnt1 is about 10GB but because its making them large on /mnt2 i'm unable to copy the entire contents.

Any help would be great.
Thank you.

[Viktor Balogh · Accepted solution]

how did you copy it? I think these are sparse files, that's why you end up with much bigger files on /mnt2.

use dd or pax to move them, see also this thread:

http://h30499.www3.hp.com/t5/System-Administration/How-can-I-copy-a-sparse-file/m-p/4859869#M396243

[Viktor Balogh]

here is a way of identifying a sparse file:

http://h30499.www3.hp.com/t5/System-Administration/Find-sparse-file/m-p/3875418#M279127

[user001]

i tried dd but it never ended :(

dd if=/dev/vg00/lvol1 of=/dev/vg01/lvol2 vs=1024

this seem to go forever, the new partition is slightly larger but that shouldn't be a problem. Is there a way to monitor dd?

i have tried cpio, mv, cp, fbackup, snapshot + those commands (in case).

i wasn't aware of pax, i noticed someone else commented that it doesn't support files larger than 2GB?

Thanks for the help.

[Viktor Balogh]

> i tried dd but it never ended :(

or you just couldn't wait till the end of it. :)
you can also copy it on a file level like this:

# dd if=/mnt1/file1 of=/mnt2/file1 bs=1024

AFAIK dd can handle files >2gig

[user001]

I left it run over night, i thought this would have been enough for dd.

I'll give pax a try first, this sounds like it could do the job. failing that i'll try dd again.

i'll update the thread / points tomorrow when i get to try it out.

thanks for the suggestions.

[Dennis Handly]

>I've noticed when copying an 800mb tar archive from /mnt1 it reports the file is 2GB on /mnt2.

Hmm. I wouldn't expect tar to be smart enough to create sparse files?

>I noticed someone else commented that it doesn't support files larger than 2GB?

No, pax(1) can support > 8 Gb. After all, after fbackup(1m) is obsoleted, pax is its replacement.

[Dennis Handly]

>I tried dd(1) but it never ended :(
dd if=/dev/vg00/lvol1 of=/dev/vg01/lvol2 vs=1024

Your blocksize is too small. Try bs=1024k

[Bob_Vance]

Hmmm.
From my testing, it seems that neither 'tar' , 'fbackup', nor pax support sparse files.
I'm surprised a little.

Just for grins, why don't you untar (watch out for absolute pathnames -- be sure to use the -C option) the tar archive into /mnt2 and see what the stuff *looks* like and actually *is*.


bv

[Hein van den Heuvel]

Can we take a few steps back and get a clear problem statement with details?


>> I'm trying to copy all the data from /mnt1 to /mnt2,

Which tool(s) used for the copy? cp? details?

>> i've noticed when copying an 800mb tar archive from /mnt1 it reports the file is 2GB on /mnt2.

Best I know there is no tool called 'it' on HPUX, although that would be neat to maximize confusion!

So what tool did you use to report the size? ls? du? df? options used for those?
Did you try alternatives? What did they all report?

hth,
Hein.



All files copied from /mnt1 to /mnt2 seem to have this problem.

[user001]

Viktor Balogh is the champion, 100% sparse files they were and pax worked a treat.

I only half expected it to work but when it did wow was i happy.

thanks for everyones help much appreciated.

cheers.

[Bob_Vance]

Wow.
That's interesting.
When I tested it, as I said, I could not get 'pax', 'tar', or 'fbackup' to *CREATE* an archive of sparse files that was small.
A 'du' of the archive was always as large as the sparse files showed in 'll', not 'du'.

I did the testing on both 11.23 and 11.31.

HOWEVER, when I *extracted* from the 'pax' archive, it DID restore a sparse file !!! ('tar' did not), even though the archive itself was large !?!?


So, I'm wondering how your archive got created as such a small size in the first place.

bv


((((((((((((((((((((((((((((((

Here is my testing:

create a sparse file:

## dd if=/dev/zero of=/tmp/sparsefile seek=1024000 count=1 bs=1024


Check sizes:

## ll /tmp/sparsefile ; du -k /tmp/sparsefile
-rw-r--r-- 1 root sys 1048577024 Apr 18 08:41 /tmp/sparsefile
8 /tmp/sparsefilep/sparsefile

So, 'll' reports 1G, while the actual 'du' size is 8K


Create archives with 'pax', 'tar', 'fbackup' :

## (cd /tmp ; pax -w -f pax1 sparsefile)
## (cd /tmp ; tar cf tar1 sparsefile)
## (cd /tmp ; fbackup -f fb1 -i sparsefile)


Check sizes:

## ( for F in /tmp/pax1 /tmp/tar1 /tmp/pax1
do ll $F; du -k $F
done
)
-rw-r--r-- 1 root sys 1048586240 Apr 18 09:10 /tmp/pax1
1032440 /tmp/pax1
-rw-r--r-- 1 root sys 1048586240 Apr 18 09:11 /tmp/tar1
1024016 /tmp/tar1
-rw-r--r-- 1 root sys 1048586240 Apr 18 09:10 /tmp/pax1
1032440 /tmp/pax1

All archives show 1G with BOTH 'll' and 'du'.

Now, extract:

## mkdir /tmp/td
## (cd /tmp/td ; pax -rv < /tmp/pax1 ; ll sp* ; du -sk sp* ; rm sp*)
USTAR format archive
sparsefile
-rw-r--r-- 1 root sys 1048577024 Apr 18 09:09 sparsefile
8 sparsefile

So, the recovered file *is* sparse, even though the archive was 1G !?!?!?!

)))))))))))))))))))))))))))))))))))))))))