Compress is slow

[Uday_S_Ankolekar]

We have a perl script that takes care of hot backup for Oracle DB. This script compress the db files and then cp it to other device.
The compression of files takes very long time.
I need to know why is it running very slow and how we resolve it.

[Patrick Wallek]

More details are needed for us to be able to adequately answer you.

How big is the dump file that you are compressing?
How long does it actually take? ("A very long time is not very helpful, sorry).
What kind of machine is this running on?

[Uday_S_Ankolekar]

Here is few info.

The Script runs on a N box with HP-UX 11.x.
Approx dump size is of 240 GB. This usually used take 7 hours, Now it runs for 14 hrs some time! There was no system changes in recent.

[Kenneth Platz]

Uday,

I can think of a few things that might speed up the compression of a 240GB file.

1) What command are you using to perform the compression? If you are using compress, you may wish to consider using gzip instead. Also, if you are using "gzip -9", you may wish to instead try using "gzip" without any options -- gzip -9 is noticeably slower than a straight gzip.

2) Are you doing a gzip followed by a copy? If that is the case, then you may wish to do the following:

# gzip < infile > /path/to/other/device/outfile.gz

This will spread I/O over two separate devices and also significantly reduce the amount if I/O this has to do.

I hope this helps.

[Uday_S_Ankolekar]

Hello:

The command being used is 'compress'without any syntex. The script compress each db file and then copy it over.
I'll try with gzip. Is there any syntex you can recommend for gzip..?

[Bill Hassell]

Just a note about compression. In general, you can't easily predict how long it will take as it is almost completely dependent on the data. So it is entirely possible for compression to take 2x-4x longer as the data changes.

Databases are usually preallocated chunks of space and compression is not only fast but VERY effective (10:1 up to 50:1) when it is empty. Fill the database with random records and the ratio drops while the compression time goes up. gzip is a good choice as there are more options to control behavior.

[Uday_S_Ankolekar]

Thanks for the resonces. I rebooted the box and backup was finished in less then 6 hours.!
Script is still using compress command!

This script runs over on a N box with 11.x and Oracle 8.1. and DLT is the device
Any clue why reboot made hot-backup faster ..?

[John Palmer]

Compression is a very cpu intensive activity. Could you have been suffering from looping rogue processes eating cpu?

Regards,
John

[Volker Borowski]

HI Uday,

I think if you have a multi CPU box, and your job uses concurrent compression in parallel, one of your CPUs might have been blocked.

As stated before, compressing is CPU intensive. If you are doing a hot backup (database keeps running), i found gzip to eat more cpu than compress, although it mostly does a better compression than compress. So if you use gzip, I would recommend to use less processes in parallel, to keep some cpu for your application.
Since gzip has a better compression, you might benefit in terms of network load for your backup. I guess you have to try out, which one is more suitable for you.

Just my 2 ?
Volker

[Volker Borowski]

... another option might be, that someone overruled your density on the DLT and set Hardwarecompression to yes (The reboot fixed this with a drive reset). Now when you use SW compression, you should NOT use any hardwarecompression (will slow down and blow up backup in size again).

On the other hand: Why do you compress at all ? DLT Hardwarcompression should do fine, be much faster and takes CPU-load from your box.

(and 0.20 more cents :-)
Volker