cancel
Showing results for 
Search instead for 
Did you mean: 

Compress is slow

Uday_S_Ankolekar
Honored Contributor

Compress is slow

We have a perl script that takes care of hot backup for Oracle DB. This script compress the db files and then cp it to other device.
The compression of files takes very long time.
I need to know why is it running very slow and how we resolve it.
Good Luck..
10 REPLIES
Patrick Wallek
Honored Contributor

Re: Compress is slow

More details are needed for us to be able to adequately answer you.

How big is the dump file that you are compressing?
How long does it actually take? ("A very long time is not very helpful, sorry).
What kind of machine is this running on?
Uday_S_Ankolekar
Honored Contributor

Re: Compress is slow

Here is few info.

The Script runs on a N box with HP-UX 11.x.
Approx dump size is of 240 GB. This usually used take 7 hours, Now it runs for 14 hrs some time! There was no system changes in recent.
Good Luck..
Kenneth Platz
Esteemed Contributor

Re: Compress is slow

Uday,

I can think of a few things that might speed up the compression of a 240GB file.

1) What command are you using to perform the compression? If you are using compress, you may wish to consider using gzip instead. Also, if you are using "gzip -9", you may wish to instead try using "gzip" without any options -- gzip -9 is noticeably slower than a straight gzip.

2) Are you doing a gzip followed by a copy? If that is the case, then you may wish to do the following:

# gzip < infile > /path/to/other/device/outfile.gz

This will spread I/O over two separate devices and also significantly reduce the amount if I/O this has to do.

I hope this helps.

I think, therefore I am... I think!
Uday_S_Ankolekar
Honored Contributor

Re: Compress is slow

Hello:

The command being used is 'compress'without any syntex. The script compress each db file and then copy it over.
I'll try with gzip. Is there any syntex you can recommend for gzip..?
Good Luck..
Bill Hassell
Honored Contributor

Re: Compress is slow

Just a note about compression. In general, you can't easily predict how long it will take as it is almost completely dependent on the data. So it is entirely possible for compression to take 2x-4x longer as the data changes.

Databases are usually preallocated chunks of space and compression is not only fast but VERY effective (10:1 up to 50:1) when it is empty. Fill the database with random records and the ratio drops while the compression time goes up. gzip is a good choice as there are more options to control behavior.


Bill Hassell, sysadmin
Uday_S_Ankolekar
Honored Contributor

Re: Compress is slow

Thanks for the resonces. I rebooted the box and backup was finished in less then 6 hours.!
Script is still using compress command!

This script runs over on a N box with 11.x and Oracle 8.1. and DLT is the device
Any clue why reboot made hot-backup faster ..?
Good Luck..
John Palmer
Honored Contributor

Re: Compress is slow

Compression is a very cpu intensive activity. Could you have been suffering from looping rogue processes eating cpu?

Regards,
John
Volker Borowski
Honored Contributor

Re: Compress is slow

HI Uday,

I think if you have a multi CPU box, and your job uses concurrent compression in parallel, one of your CPUs might have been blocked.

As stated before, compressing is CPU intensive. If you are doing a hot backup (database keeps running), i found gzip to eat more cpu than compress, although it mostly does a better compression than compress. So if you use gzip, I would recommend to use less processes in parallel, to keep some cpu for your application.
Since gzip has a better compression, you might benefit in terms of network load for your backup. I guess you have to try out, which one is more suitable for you.

Just my 2 ?
Volker
Volker Borowski
Honored Contributor

Re: Compress is slow

... another option might be, that someone overruled your density on the DLT and set Hardwarecompression to yes (The reboot fixed this with a drive reset). Now when you use SW compression, you should NOT use any hardwarecompression (will slow down and blow up backup in size again).

On the other hand: Why do you compress at all ? DLT Hardwarcompression should do fine, be much faster and takes CPU-load from your box.

(and 0.20 more cents :-)
Volker
Uday_S_Ankolekar
Honored Contributor

Re: Compress is slow

hello Volker,

The script does compress the db files then move over it to a directory (say a backup directory )within the system. From the script log I can make out the compressing is taking much longer time.

Good Luck..