topic Re: gzip a large file question in Operating System - HP-UX

gzip a large file question

Bruce Duff — Thu, 25 Apr 2002 20:28:04 GMT

Hi there, I am trying to gzip up a large file of 3.7gb. It is an export of my Oracle dbase. When I gzip it, it returns :

unknown error, (this is very unhelpful), has anyone else out there seen this before and how can I work out what is going wrong.

Thanks
Bruce

Re: gzip a large file question

someone_4 — Thu, 25 Apr 2002 20:33:45 GMT

Hi,
There is 2 gig limit. Here is the doc.

http://us-support.external.hp.com/cki/bin/doc.pl/sid=f4db301b102edc718a/screen=ckiDisplayDocument?docId=200000059774342

- Richard

Re: gzip a large file question

hpuxrox — Thu, 25 Apr 2002 20:38:22 GMT

I suggest downloading and installing the latest GNU gzip from gnu.org, there is a 2G limit on the default HPUX version.

Re: gzip a large file question

SHABU KHAN — Thu, 25 Apr 2002 21:36:31 GMT

Bruce,

Have you tried splitting it using the split command and then gzip'ng it ?

split -b oracle_datfile

I would suggest splitting it by half ..

then:

mv xaa oracle_datfile_part1
mv xab oracle_datfile_part2

gzip oracle_datfile*

--------
Later to merge these ... just do:

gzip -d oracle_datfile*
cat oracle_datfile* > oracle_datfile

I haven't done this before, just a thought ...

Take a backup of this dat file if you have enough space, else don't try this method.

PS. Try only if you have enough space to back-up or if you cannot get the GNU zip for other company policy reasons

Thanks,
Shabu

Re: gzip a large file question

Deepak Extross — Fri, 26 Apr 2002 01:00:46 GMT

Try using compress and see if it works.

Re: gzip a large file question

T. M. Louah — Fri, 26 Apr 2002 01:39:07 GMT

Check the HP-UX portal the list contains the latest gzip binaries that supports files larger than 2GB.
http://hpux.cs.utah.edu/hppd/hpux/Gnu/

here's a good discussion about it :
http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x426c50011d20d6118ff40090279cd0f9,00.html

Remember to ASIGN POINTS !!!!!!!!! :)

Re: gzip a large file question

Ulf Lipski — Fri, 26 Apr 2002 05:50:30 GMT

Hi,

why do you want to zip this export file?

Note: In newer Oracle-Versions (I just can't remember the exact release, somtething beyond 8.X) you can export into several files!

Bye

Re: gzip a large file question

Yogeeraj_1 — Sat, 27 Apr 2002 10:19:42 GMT

Re: gzip a large file question

harry d brown jr — Sat, 27 Apr 2002 10:32:35 GMT

bruce duff - This member has assigned points to 6 of 69 responses to his/her questions.

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x96b70bce6f33d6118fff0090279cd0f9,00.html
http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x316efd3f91d3d5118ff40090279cd0f9,00.html

Great help in finding them:
http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x1e6f84534efbd5118ff40090279cd0f9,00.html

and get the newest gzip here (mar 2002)
http://hpux.cs.utah.edu/hppd/hpux/Gnu/gzip-1.3.3/

live free or die
harry

Re: gzip a large file question

Alberto Minichiello — Sat, 27 Apr 2002 20:18:59 GMT

Hi:

As everyone else mentioned already, gzip on HP-UX 11.00
is a 32 bit (PA-Risc 1.1) binary,
as far as I know it can't handle file sizes over 2GB.

Here is my 2 cents worth:

For the file you already have, use the following command:

dd if=Original_File bs=8192k | gzip -c | dd of=compressed_file bs=8192k

this should do the trick, the only downside is that of course the original file is not deleted as gzip usually does.

My Oracle exports are always compressed cause they are usually over 15 GB in which case I do this to have export compress it directly.

1) Create a FIFO (with mknod)

2) Make sure that your output file on exp.par points to that fifo.

3) Before running exp put a command similar to the one above on background, like this:

dd if=fifo_file bs=8192k | gzip -c | dd of=destination_file bs=8192k

4) run exp

this will export the database and compress in one fell swoop.

I find it very handy cause reading a 30 GB monster of tape can take a while just to recover a table.

BTW the compress ratio I get is 1/3.

Regards

Daniel Cristini

Re: gzip a large file question

Choi Eun Kyoung — Sun, 28 Apr 2002 23:36:30 GMT

You can confirm the gzip software version by "gzip -V" command.
# gzip -V
If the gzip version under 1.2.4, It don't support large file. but If the gzip version over 1.2.4A, It can handle large files(over 2GB).
and You can get the newest version from the below site.
http://hpux.cs.utah.edu/hppd/hpux/Gnu/gzip-1.3.3/

Re: gzip a large file question

Bruce Duff — Tue, 30 Apr 2002 08:27:21 GMT

Thanks everyone for your replies. There are two different apporaches to this problem.

However , the real cause I feel was the old version of gzip installed on the server. 1.2.4 which couldnt cope with files > than 2gb. If you go to the hp software archive you can dowload the latest version which copes very well with large files. My export was 3.7gb. This issue has now been resolved.

Thanks for your help again.

All the best
Bruce