how to make TAR to run faster

[Steven Chen_1]

Hi,

I have 7G data needed badly to extract from a 12G tape drive. It runs forever. The server is K260 with 2 cpus. Not much other processes going on.

I guess there is some command to make tar to be running faster.

Thanks.

[Patrick Wallek]

As far as I know there is not much way to speed up tar. The only thing I can think of that might help a little is to NOT use the verbose '-v' option when extracting. Other than that you'll just have to let it run. It will finish eventually.

[A. Clay Stephenson]

Hi Steven,

Try this, we are going to use dd to icease the input blocking:

dd if=/dev/rmt/0m ibs=2000b obs=1b | tar xvf -

This will reblock input to 2000 512 byte blocks.

You may need to adjust the obs to perhaps 20b depending on the original tar blocking factor.

Before you actually extract, I would pipe the dd to a tar vtf - to attempt a listing first. That way you can adjust the obs setting until you get consistant results. You can kill (or interrupt) the process until you get the settings right. Again, i would start with obs=1b, then try 20b.

This should speed up the process by at least 30x.

Clay

[Steven Chen_1]

Patric,

Thanks for the help.

Would "tar x ...." just enough?
or "tar xf ..." if not use xvf?

I am using xvf that is as slow as whatever.

[John Poff]

If the tape drive and the disk drive are on the same SCSI bus I guess there could be a traffic problem as you might be reading and writing the data on the same bus. The only way I know to get around that is to somehow read the tape from another system and send the output through the LAN. I've thought about doing that before but I've never tried it.

[Patrick Wallek]

Yes you could use 'tar -x filetorestore' or 'tar xf /dev/rmt/?m filetorestore'. The 'v' option is what causes everything to display to your terminal, so it could slow you down.

Clays solution looks interesting. I may have to play with that myself.

[Steven Chen_1]

Clay,

Thank you for the thought.

Those 7G data are Oracle datafiles used to recover database. I would need to make sure any method would not cause data corruption.

I was using tar to backup, so I use tar to extract.

[Steven Chen_1]

another question related:

I use oracle user to tar cvf file. While I tar xf extracting file, would I have to use oracle user too? Would root account run faster? Or that would change the file bits?

[Carlos Fernandez Riera]

Talking about blokages:

Use tar cvbf 64 /dev/rmt ...

Or tyr pax.

[Bill Hassell]

tar is not in itself the problem unless your machine is very slow (less than 50 Mhz). tar reads the file as fast as the opsystem allows and outputs the result to the tape. To see just how fast tar is, change the output tape device to /dev/null...it should complete in a very short time.

So the problem is getting the data to the tape fast enough to prevent data starvation. Like all streaming drives, if the data does not keep ahead of the device, the drive stops, backs up and resyncs, a very costly task (wastes a lot of time).

The tape drive must be on a dedicated bus or on a bus with no activity on any other connected peripheral. If the tape and the disk are on the same bus, there is nothing you can do to improve performance.

It's also important to note that tar (pax, cpio, dump) is NOT recommended for large data backup tasks. It is a couple of decades old and designed for reel-to-reel tape drives holding a whopping 150 megs. Industry standard tar can never handle a file bigger than 2 Gb and has no error recovery procedures. If this data is important, strongly consider a commercial quality backup program. Or as a minimum, use fbackup which does handle large files and has error recovery built-in. Do not use fbackup without a config file to set large records and a large number of helper proceses.