compress fails with I/O error

[Ian Lochray]

I have a 16Gb tar file which I am trying to compress with "compress filename". It works for a while then fails with an I/O error when the compressed file has reached approximately 2.5Gb.
Am I hitting some king of file size limit for compress? Note that the 16Gb tar file is in the same directory and I am using the same user I user to create the tar file.

[Pete Randall]

Is there sufficient room to contain both files?


Pete

[Ian Lochray]

There is plenty of space for both files.

[Ian Lochray]

I have now tried the compress again and it fails when the compressed file gets to exactly the same size. Looks like some kind of limit to me.

[Peter Nikitka]

Hi,

first check, if you find something related in the syslog or dmesg-outpout.
Then try
dd if=fromfile of=/dev/null bs=1024k

If this shows I/O errors as well, you have a problem with the disk the file resides.

Will the compress output be created in the same filesystem/disk?
If no, just copy the file to the destination target - you said, there is plenty of space...

Else:
Perhaps the compress is not able to handle more than 2GB? This would be 2147483648 Bytes.

Try other shrink utilities (if installed):
gzip to.gz
bzip2 to.bz
zip from

mfG Peter

[Steven Schweda]

I know nothing, but I'd bet on a 2GB size
limit, too.

> gzip to.gz

gzip 1.3.x should offer large-file support
with less fooling around.

http://www.gzip.org/
http://ftp.gnu.org/gnu/gzip/

> bzip2 to.bz

bzip2 should offer large-file support.

http://www.bzip.org/

> zip from

(Is that a valid Zip command?) Large-file
support in the Info-ZIP programs doesn't
appear until Zip 3.0 and UnZip 6.0, which
haven't been released yet. Imperfect but
usable pre-release "BETA" source kits exist
at:

ftp://ftp.info-zip.org/pub/infozip/beta

[Ian Lochray]

It wasn't a 2Gb file limit as the file got to 2.5Gb.
In the end I recreated by 16Gb tar file as 4 x 4Gb files are compressed them individually