topic compressing large files in Operating System - HP-UX

compressing large files

Mukundan_1 — Tue, 18 May 2004 13:28:00 GMT

Hp-UX gurus,
i have question. do you forsee or have you experienced any problems with using large files (especially oracle data files) of sizes 60gb+?

how much free space do we need to compress a file of size 60gb so that the compression proceeds without running out of disk space?
i appreciate your response.

thanks
Mukundan

Re: compressing large files

Cesare Salvioni — Tue, 18 May 2004 13:32:15 GMT

I never used files so large but the command compress needs free space equal to the file size, when it gets this size for the compressed file it stops
Using gzip should be better but i don't know how much it needs

Hope this help
Cesare

Re: compressing large files

Paula J Frazer-Campbell — Tue, 18 May 2004 13:53:07 GMT

Hi

During the compression process of very large files disk space is used as a buffer/build area so as a general rule twice the file size is required.

Paula

Re: compressing large files

Juergen Tappe — Tue, 18 May 2004 14:03:26 GMT

That depends on how good the file is compressable.

During compress or gzip "file.Z" or "file.gz" growes and as soon the compress is finished "file" gets removed.

If you have a compression rate of 90% you would need 6 GB.
By a compression rate of 50% you would need 30 GB.
And if the file is not compressable at all you need the whole 60 GB free.

To be sure you should have minimum the same size as the original file free.

Re: compressing large files

Ryan B — Tue, 18 May 2004 14:15:44 GMT

Hey Mukundan.

If you decide to use gzip, make sure you have the proper version or it won't compress files bigger than 2GB. The version that is shipped with HPUX does not support it and you have to get a later version form the HP Portal site to support 2GB or larger files...

Re: compressing large files

Dani Seely — Tue, 18 May 2004 14:27:03 GMT

Hey Mukundan,
You can use 'compress' or 'gzip'. If you have problems with disk space on the drive where your large files are stored, you should find a filesystem with available free space and create an archive directory to move the files to and then compress them there.

You can set up a crontab entry to save off and compress the large files based on their age and then move the file to the archive directory. The commands you would set up in the crontab woud be something like:
find -mtime +1 -exec compress -f {} \;
mv *.Z

Re: compressing large files

Dani Seely — Tue, 18 May 2004 22:48:20 GMT

Hello Mukundan,
I assume this is your first experience on the ITRC forum as you did not award points to the forumers for the answers you were provided. May I suggest that you take a look at the following link to learn about the points system in use here. Thanks.

http://forums1.itrc.hp.com/service/forums/helptips.do?#28

Please read the article, assess the assistance you were provided by the forumers, then reward them. Thanks!

Re: compressing large files

KapilRaj — Wed, 19 May 2004 01:14:58 GMT

It depends on what u r trying to compress.
when compress is invoked on a text file, u normally get a 75-80 % compression.

So u can successfully compress a text file if you have 30 %(safer side) of the original text file, space left in ur filesystem.

Regds,

Kaps

Re: compressing large files

Nicolas Dumeige — Wed, 19 May 2004 02:36:28 GMT

Hello,

60 Gb is larger than the average dbf. On what basis did you pick this size ? How large is you database ? Our is 700 GB datawarehouse and the production standard is 5 Gb dbf. I don't know what is the recommandation from Oracle though ...

Cheers

Nicolas