topic Incorrect file size reported in Operating System - HP-UX

Incorrect file size reported

user001 — Fri, 15 Apr 2011 05:04:15 GMT

Hello,
I have the following disk layout
vg00
lvol1
/mnt1
vg01
lvol2
/mnt2

I'm trying to copy all the data from /mnt1 to /mnt2, i've noticed when copying an 800mb tar archive from /mnt1 it reports the file is 2GB on /mnt2. All files copied from /mnt1 to /mnt2 seem to have this problem.

11.31
vxfs v6
20GB lv's

Any ideas. This is driving me insane, /mnt1 is about 10GB but because its making them large on /mnt2 i'm unable to copy the entire contents.

Any help would be great.
Thank you.

Re: Incorrect file size reported

Viktor Balogh — Wed, 17 Aug 2011 15:45:47 GMT

how did you copy it? I think these are sparse files, that's why you end up with much bigger files on /mnt2.

use dd or pax to move them, see also this thread:

http://h30499.www3.hp.com/t5/System-Administration/How-can-I-copy-a-sparse-file/m-p/4859869#M396243

Re: Incorrect file size reported

Viktor Balogh — Mon, 11 Jul 2011 21:33:06 GMT

here is a way of identifying a sparse file:

http://h30499.www3.hp.com/t5/System-Administration/Find-sparse-file/m-p/3875418#M279127

Re: Incorrect file size reported

user001 — Fri, 15 Apr 2011 07:55:09 GMT

i tried dd but it never ended :(

dd if=/dev/vg00/lvol1 of=/dev/vg01/lvol2 vs=1024

this seem to go forever, the new partition is slightly larger but that shouldn't be a problem. Is there a way to monitor dd?

i have tried cpio, mv, cp, fbackup, snapshot + those commands (in case).

i wasn't aware of pax, i noticed someone else commented that it doesn't support files larger than 2GB?

Thanks for the help.

Re: Incorrect file size reported

Viktor Balogh — Fri, 15 Apr 2011 08:03:17 GMT

> i tried dd but it never ended :(

or you just couldn't wait till the end of it. :)
you can also copy it on a file level like this:

# dd if=/mnt1/file1 of=/mnt2/file1 bs=1024

AFAIK dd can handle files >2gig

Re: Incorrect file size reported

user001 — Fri, 15 Apr 2011 08:06:05 GMT

I left it run over night, i thought this would have been enough for dd.

I'll give pax a try first, this sounds like it could do the job. failing that i'll try dd again.

i'll update the thread / points tomorrow when i get to try it out.

thanks for the suggestions.

Re: Incorrect file size reported

Dennis Handly — Sat, 16 Apr 2011 11:12:22 GMT

>I've noticed when copying an 800mb tar archive from /mnt1 it reports the file is 2GB on /mnt2.

Hmm. I wouldn't expect tar to be smart enough to create sparse files?

>I noticed someone else commented that it doesn't support files larger than 2GB?

No, pax(1) can support > 8 Gb. After all, after fbackup(1m) is obsoleted, pax is its replacement.

Re: Incorrect file size reported

Dennis Handly — Sat, 16 Apr 2011 11:13:51 GMT

>I tried dd(1) but it never ended :(
dd if=/dev/vg00/lvol1 of=/dev/vg01/lvol2 vs=1024

Your blocksize is too small. Try bs=1024k

Re: Incorrect file size reported

Bob_Vance — Sun, 17 Apr 2011 12:30:03 GMT

Hmmm.
From my testing, it seems that neither 'tar' , 'fbackup', nor pax support sparse files.
I'm surprised a little.

Just for grins, why don't you untar (watch out for absolute pathnames -- be sure to use the -C option) the tar archive into /mnt2 and see what the stuff *looks* like and actually *is*.

bv

Re: Incorrect file size reported

Hein van den Heuvel — Sun, 17 Apr 2011 13:49:14 GMT

Can we take a few steps back and get a clear problem statement with details?

>> I'm trying to copy all the data from /mnt1 to /mnt2,

Which tool(s) used for the copy? cp? details?

>> i've noticed when copying an 800mb tar archive from /mnt1 it reports the file is 2GB on /mnt2.

Best I know there is no tool called 'it' on HPUX, although that would be neat to maximize confusion!

So what tool did you use to report the size? ls? du? df? options used for those?
Did you try alternatives? What did they all report?

hth,
Hein.

All files copied from /mnt1 to /mnt2 seem to have this problem.

Re: Incorrect file size reported

user001 — Mon, 18 Apr 2011 08:14:13 GMT

Viktor Balogh is the champion, 100% sparse files they were and pax worked a treat.

I only half expected it to work but when it did wow was i happy.

thanks for everyones help much appreciated.

cheers.

Re: Incorrect file size reported

Bob_Vance — Mon, 18 Apr 2011 12:48:33 GMT

Wow.
That's interesting.
When I tested it, as I said, I could not get 'pax', 'tar', or 'fbackup' to *CREATE* an archive of sparse files that was small.
A 'du' of the archive was always as large as the sparse files showed in 'll', not 'du'.

I did the testing on both 11.23 and 11.31.

HOWEVER, when I *extracted* from the 'pax' archive, it DID restore a sparse file !!! ('tar' did not), even though the archive itself was large !?!?

So, I'm wondering how your archive got created as such a small size in the first place.

bv

((((((((((((((((((((((((((((((

Here is my testing:

create a sparse file:

## dd if=/dev/zero of=/tmp/sparsefile seek=1024000 count=1 bs=1024

Check sizes:

## ll /tmp/sparsefile ; du -k /tmp/sparsefile
-rw-r--r-- 1 root sys 1048577024 Apr 18 08:41 /tmp/sparsefile
8 /tmp/sparsefilep/sparsefile

So, 'll' reports 1G, while the actual 'du' size is 8K

Create archives with 'pax', 'tar', 'fbackup' :

## (cd /tmp ; pax -w -f pax1 sparsefile)
## (cd /tmp ; tar cf tar1 sparsefile)
## (cd /tmp ; fbackup -f fb1 -i sparsefile)

Check sizes:

## ( for F in /tmp/pax1 /tmp/tar1 /tmp/pax1
do ll $F; du -k $F
done
)
-rw-r--r-- 1 root sys 1048586240 Apr 18 09:10 /tmp/pax1
1032440 /tmp/pax1
-rw-r--r-- 1 root sys 1048586240 Apr 18 09:11 /tmp/tar1
1024016 /tmp/tar1
-rw-r--r-- 1 root sys 1048586240 Apr 18 09:10 /tmp/pax1
1032440 /tmp/pax1

All archives show 1G with BOTH 'll' and 'du'.

Now, extract:

## mkdir /tmp/td
## (cd /tmp/td ; pax -rv < /tmp/pax1 ; ll sp* ; du -sk sp* ; rm sp*)
USTAR format archive
sparsefile
-rw-r--r-- 1 root sys 1048577024 Apr 18 09:09 sparsefile
8 sparsefile

So, the recovered file *is* sparse, even though the archive was 1G !?!?!?!

)))))))))))))))))))))))))))))))))))))))))