Find sparse file?

[Prashant Zanwar_4]

Any way of finding sparse file?
Thanks and rgds
Prashant

[A. Clay Stephenson · Accepted solution]

Bear in mind that the fact that a file is sparse is intentionally made invisible to the system except in a few cases. For example, when you copy the file the read() system call will silently fill in the "missing" data with ASCII NUL's. The only way to know if a file is sparse is to look at its size as reported by stat() (which ls -l uses) and then compare it to the output of "du -k" (which looks at the underlying filesystem usage). If these two values differ widely then the files contains sparse data.

Do something like this:

tr "\000" "x" < /dev/zero | dd bs=1 count=1 of=x1 # write 'x' at offset 0
tr "\000" "x" < /dev/zero | dd bs=1 count=1 oseek=999999 of=x1 # write 'x' at offset 999999

x1 is now a sparse 1MB file.

Now let's create a normal 1MB file (I'm doing this inefficiently so it will take a little time but this will closely mimic the way the sparse file was created for consistancy):

tr "\000" "x" < /dev/zero | dd bs=1 count=1000000 of=x2

Now do an "ls -l x1 x2"
and compare that to a "du -k x1 x2".

Note: du -k reports the file size in 1024 byte blocks.

[Bill Hassell]

The normal uses for sparse files are with database programs. The database code will aloocate a fixed amount of space in the filesystem but not actually write any records for a while. Although the file does not actually occupy all the space, it is still part of the file. So when the file is read serially, the undefined records are created by the filesystem code as a series of nulls.

Now frecover has an option to 'handle' sparse files on an fbackup tape, but what it really does is to convert long strings of nulls into lseek() operations to skip these null blocks. This minimizes the amount of space needed for a sparse file or perhaps a file that was never stored as sparse (actually full of nulls) but restores with less space.

[A. Clay Stephenson]

If you really want to find them then it's rather easy. The attached script, sparsechk.sh, will do an ls -l / 1024 and a du -k and if the values differ by more than 3 blocks, the filename is output.

Use it like this:

find . -type f | xargs sparsechk.sh

It won't be particularly fast but it will work.

[Sandman!]

See the TKB doc below:

http://www1.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000077186715