topic Re: Compression of Big files in Operating System - HP-UX

Compression of Big files

Vibhor Kumar Agarwal — Wed, 14 Sep 2005 05:20:07 GMT

Hi,

I have a 3 GB and bigger tar files.

What are the ways of compressing it.
While using gzip it says, file too large.

Re: Compression of Big files

Muthukumar_5 — Wed, 14 Sep 2005 05:25:06 GMT

You can use bzip2 for this purpose.

Refer this: http://www.digistar.com/bzip2/

hth.

Re: Compression of Big files

Arunvijai_4 — Wed, 14 Sep 2005 05:27:34 GMT

1) You can try with bzip2
2) Compress the file in a location where you have free space more than 3 GB.
3) Use gzip -9 (Better compression,bit slow)

-Arun

Re: Compression of Big files

Vibhor Kumar Agarwal — Wed, 14 Sep 2005 05:36:07 GMT

I'll try bzip2 for this.

gzip -9 gives the same output.

This time i have copied it.
"Value too large for defined data type"

Re: Compression of Big files

Vibhor Kumar Agarwal — Wed, 14 Sep 2005 05:38:28 GMT

No I can't bzip2.

Because i have to deliver this to different locations, and they won't install that.

Re: Compression of Big files

VEL_1 — Wed, 14 Sep 2005 05:40:45 GMT

Hi,

Which file system are u using? And OS?

Re: Compression of Big files

Arunvijai_4 — Wed, 14 Sep 2005 05:41:02 GMT

I think, you need to compile gzip to enable large file compression.

http://www.brandonhutchinson.com/gzip__Value_too_large_for_defined_data_type.html

Re: Compression of Big files

Arunvijai_4 — Wed, 14 Sep 2005 05:42:41 GMT

One more link, http://www.gzip.org/#faq10

Re: Compression of Big files

VEL_1 — Wed, 14 Sep 2005 05:50:23 GMT

Alternatively, you can try to use the dd command:

Please see the following link:

http://wdb1.sco.com/kb/showta?taid=112962

Note:

If you used old version of gzip, its better to upgrade gzip. Because old version of gzip may not support the large files.

Re: Compression of Big files

Vibhor Kumar Agarwal — Wed, 14 Sep 2005 05:52:16 GMT

Just a small problem:

I am not an administrator, so don't have enough rights to do the installation.

Re: Compression of Big files

HugoAlmeida — Wed, 14 Sep 2005 05:53:21 GMT

Hi

Try
compress -fv

HTH

Re: Compression of Big files

Arunvijai_4 — Wed, 14 Sep 2005 05:53:42 GMT

Can you compile gzip yourself on your server ? Do you have enough permission to do that ?
In such cases, you can build your own gzip and use it for compression.

Re: Compression of Big files

Alessandro Pilati — Wed, 14 Sep 2005 06:17:02 GMT

Download gzip here and recompile it http://hpux.connect.org.uk/hppd/hpux/Gnu/gzip-1.3.5/
without restrictions

or
download this package:
http://h20293.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=HPUXIEXP1123

regards,
Alex

Re: Compression of Big files

Vibhor Kumar Agarwal — Wed, 14 Sep 2005 06:37:22 GMT

Compressing is working, but its too slow and also it doesn't give a good compression ratio.

Compiling gzip by myself will end up in the same problem, when i deliver the file, the reciepient might not be able to decompress it.

Re: Compression of Big files

Vibhor Kumar Agarwal — Wed, 14 Sep 2005 07:01:52 GMT

Hey,

That dd option seems to work. :-)
I just replaced compress with gzip and is working.
Will take some time to complete, will update you then.

Re: Compression of Big files

Vibhor Kumar Agarwal — Wed, 14 Sep 2005 07:53:51 GMT

Wow,

That dd worked wonderfully.

At the end of dd it gave me the output:
300+1 records in
300+1 records out
0+52264 records in
0+52264 records out

What does it mean.

Re: Compression of Big files

Fabio Ettore — Wed, 14 Sep 2005 07:59:54 GMT

Hi,

what dd command did you launch exactly which got that output?

Best regards,
Fabio

Re: Compression of Big files

Vibhor Kumar Agarwal — Wed, 14 Sep 2005 23:40:02 GMT

Check out the raj's link.

Re: Compression of Big files

Michael Roberts_3 — Thu, 15 Sep 2005 01:15:38 GMT

cat bigFile | gzip -c - | cat - > bigFile.gz

'cat' is OK with large files and since 'gzip' is reading/writing a pipe stream it doesn't have to deal with a large file descriptor.

Re: Compression of Big files

Vibhor Kumar Agarwal — Thu, 15 Sep 2005 23:48:21 GMT

Its not a file, its a whole directory structure tarred up.

Will it work.