Operating System - HP-UX
1826451 Members
3861 Online
109692 Solutions
New Discussion

gzip utility running on 2 or more processors

 
Ivan Efimenko
Contributor

gzip utility running on 2 or more processors

We have server with 4 processors. Standard gzip utility can run only on one processor (I've checked this by top utility).
But is it possible to run it on MORE than 1 processor?
If NOT, are there any alternatives?

Thank you in advance.
3 REPLIES 3
Pedro Sousa
Honored Contributor

Re: gzip utility running on 2 or more processors

Each process can only run on one processor.
If you're running gzip on some file/dir, it will only run on one processor at a time.
I think there's no way to change this!
good luck.
Kenneth Platz
Esteemed Contributor

Re: gzip utility running on 2 or more processors

Ivan,

As far as I know, the "gzip" utility is exclusively singlethreaded -- it can only be run on a single processor. Furthermore, I doubt whether this actually CAN be multithreaded since with "gzip" (and most other compression schemes) subsequent blocks of data are dependent upon previous blocks.

However, if you tried to run 4 copies of "gzip" (on 4 different files), each of them could realistically use a separate processor.
I think, therefore I am... I think!
Dragan Krnic
Frequent Advisor

Re: gzip utility running on 2 or more processors

gunzip can decompress chunkwise - a nice feature if you're in a hurry. It means that you can slice a huge file into smaller parts compress them with gzip on individual processors and then merge the resulting compressed chunks into a file in proper sequence. Suppose your file is 4 GB. You can do the following (debug it yourself):
for i in 0 1 2 3
do dd if=file bs=1024k skip=$(expr 1024 \* $i) count=1024 | gzip -9 > file$i &
done
After all processes finish just execute
cat file2 file3 file4 >> file1
and rename it like file.gz