Block size

[Conrad J Wyrzykowski]

Greetings,

I understand the concept of the 512 or 1024 block sizes or so I think. If while using the du -ak command the file size comes back as 280 then logic tells me that 1024 * 280 = 286720 bytes. But when I check the same file using Windows I get 184380 bytes or 192512 bytes as the size on disk which makes sense because of the block size. But 192512 does not equal 286720 why?

Thanks,
Conrad

[A. Clay Stephenson]

What does ls -l filename report and what command are you using on Windows to report the file size. Moreover, is this the same physical file (e.g Samba mounted) or did you copy the file from UNIX to Windows; if so, how? There is a piece of the puzzle that's missing.

[Ivan Ferreira]

du does not reports whell for single file, because is based on disk blocks.

For example:

ls -la
-rw-r--r-- 1 ferreiri ferreiri 119 Jan 16 18:53 test

du -ak test
1 test

The file has 119 bytes but it uses a whole disk block. It cannot use less than one full disk block.

[Conrad J Wyrzykowski]

ls -l reports 8192 using the directory above the coverage if I do it at the coverage level I get:
pfwpg02:/mbr_lib/bas/a31/4_/mb $ ls -l a314_mb
total 544
-rwxrwxr-x 1 lewisr lib 2304 Mar 18 1999 aat.adf
-rwxrwxr-x 1 lewisr lib 170772 Mar 18 1999 arc.adf
-rwxrwxr-x 1 lewisr lib 676 Mar 18 1999 arx.adf
-rwxrwxr-x 1 lewisr lib 1152 Mar 18 1999 cnt.adf
-rwxrwxr-x 1 lewisr lib 364 Mar 18 1999 cnx.adf
-rwxrwxr-x 1 lewisr lib 32 Mar 18 1999 dblbnd.adf
-rwxrwxr-x 1 lewisr lib 80 Mar 18 1999 dbltic.adf
-rwxrwxr-x 1 lewisr lib 1892 Mar 18 1999 lab.adf
-rwxrwxr-x 1 lewisr lib 1539 Mar 18 1999 log
-rwxrwxr-x 1 lewisr lib 3292 Mar 18 1999 pal.adf
-rwxrwxr-x 1 lewisr lib 260 Mar 18 1999 par.adf
-rwxrwxr-x 1 lewisr lib 924 Mar 18 1999 pat.adf
-rwxrwxr-x 1 lewisr lib 364 Mar 18 1999 pax.adf
-rwxrwxr-x 1 lewisr lib 729 Mar 18 1999 prj.adf

Which totals 184830.

In windows I'm just using windows explorer and right clicking on the file and selecting properties. Which represents what windows is indicating for the file size not the size on the disk.

[A. Clay Stephenson]

Nevertheless, du -ak sould be accurate to within 1 block and these numbers are nowhere near that. du and ls -l can also yield radically different results when reporting on sparse files. Both numbers are "correct"; they are simply answering different questions.

[A. Clay Stephenson]

Nevertheless, du -ak should be accurate to within 1 block and these numbers are nowhere near that. du and ls -l can also yield radically different results when reporting on sparse files. Both numbers are "correct"; they are simply answering different questions.

[A. Clay Stephenson]

Now I begin to understand. You are comparing apples to oranges. Your UNIX ls -l is equivalent to the Windows properties total. Du is looking at underlying filesystem usage which is not the same at all. For example, if you have a file whose contents are "I am 19 bytes long\n" then ls -l will display 19 bytes but du -ak will display at least 1 (1KB) blocks BUT it could be much higher still because the smallest possible filesystem block might be 8K for example. Moreover, if that same 19-byte string were written at offset (with no data written before it) 1,000,000 then ls -l would display 1,000,019 but du -ak would still display 1 block. It all depends on what question you are asking the system.

[Conrad J Wyrzykowski]

Okay, I think it is clear as mud. If I want to know the real size of the file I should use ls -l but if I want to know how much disk space will be consumed by the file du is my work horse.

[A. Clay Stephenson · Accepted solution]

You almost have it. You have to understand that the filesysstem blocksize refers to the smallest quantum of space used by a file. This isn't the same thing as du's 512 or 1024 byte blocks.

This means that a 19, 190, and 1024 byte file will occupy 1 filesystem block -- assuming the vxfs filesystem's 1K block size.

It really gets complicated when sparse files come into the picture.
Consider writing 'X' at offset 0 and then doing a lseek to offset 999,999 and writing another 'X'. A total of two bytes of data will have been written. Because these crossed block boundaries, 2 filesystem blocks will have been written and 2 will be ported by du -ak, but ls -l will report 1,000,000 as the file length. All of these are "correct". Sparse files also make it possible to tremendously overcommit a filesystem so that like much more data (as totaled by ls -l) might be there than the filesystem could actually hold. When sparse files are read, the "missing" data are filled in with ASCII NUL's.

[Conrad J Wyrzykowski]

Thanks for all your assistance in understanding the file size discrepancy.

Cheers,
Conrad