corrupt tar file

[lawrenzo_1]

Hello,

a developer here has created a tar file and compressed it by running the following command:

tar cvf - * |compress -c > tarfile.Z

to uncompress the file he runs:

cat tarfile.Z |uncompress |tar xvf -

the issue is when he attempts to uncompress the file there is an error:

tar: 0511-169 A directory checksum error on media; 0 not equal to 8692.

Ok so some tests I attempted:

I uncompressed the file and ran the file command:

--> file dtr01datafiles_BCK.tar
dtr01datafiles_BCK.tar: data or International Language text

states not a tar file.

I ran through the compress and uncompress process and this was successful.

The developer has assured me he has been compressing the data using this method for some time and has managed to uncompress the data in the past.

The actual file created is 44GB and when compressed is 8gb.

does anyone have any idea's how we can recover the data?

Thanks

Chris

[spex]

Hello Chris,

Are you archiving/extracting on the same system, or two systems (same OS or different OSes)? If two, are you using different versions of tar? What does 'cksum tarfile.Z' on each end tell you?

Where is the compressed tarball being stored? Disk? Tape? DVD-ROMs?

Does tar still complain if you forgo compressing/uncompressing the tarball? How about if you use gzip (http://hpux.connect.org.uk/hppd/hpux/Gnu/gzip-1.3.12/) instead of compress?

PCS

[Ivan Ferreira]

I just tested in this way, can you try that?

tar cvf - /etc/a* | compress -c > etc.tar.Z

zcat etc.tar.Z | tar tvf -

I agree that compress should be replaced by another compression utility.

[lawrenzo_1]

# cksum dtr01datafiles_BCK.tar.Z
1983363346 8311147750

so this is inconsistant ...

zcat dtr01datafiles_BCK.tar.Z |tar tvf -
tar: 0511-169 A directory checksum error on media; 0 not equal to 8692.

I think we can determine that the tarfile has become corrupt and I don't like the use of -c in this case.

I have suggested they use:

tar cvf - * |gzip > file.tar.gz
and to recover
gunzip < file.tar.gz |tar xvf -

Thanks for the help, but any suggestions on how to recover the data from the corrupt tarfile?

[spex · Accepted solution]

A quick Google search turns up Advanced Tar Repair:

http://www.tar-repair.com/

Note that I have never used this product, so I can't speak to its effectiveness.

Unfortunately, even in the best case, you will experience data loss.

[Dennis Handly]

You may want to look at the first few 1000 bytes of the uncompressed file to see how bad it is:
$ zcat tarfile.Z | xd -tx4 -tc -N 2048
Or if you still have it:
$ xd -tx4 -tc -N 2048 dtr01datafiles_BCK.tar

You then can match it up with tar(4).
http://docs.hp.com/en/B2355-60103/tar.4.html

>The actual file created is 44GB

I'm not sure if gzip handles large files yet? I suppose you could use cat and a pipe.

[Steven Schweda]

Can we assume from the unusually well
defined error message ("0511-169 [...]")
that you're actually working on an AIX
system? (In which case, asking in an HP-UX
forum may not be the most efficient way to
access relevant expertise.)

You're running large-file-sized data sets
(>2GB) through multiple programs which might
have trouble with large files. The data
could be corrupted in almost any step in the
process.

As others have suggested, "cat" with a pipe
may work better with some programs than
asking those programs to read directly from a
(large) file.

It might be helpful to get a clearer
explanation of exactly what was done where,
and if any files were transferred from one
system to another (and how), but my guess is
that you'll need to back up and see what the
limitations (file size and other) are for the
programs you're using on the systems where
you're using them.

> does anyone have any idea's [...]

Those are "ideas" (no apostrophe).

[Bill Hassell]

Was this file transferred before you tried to unpack it? Did you use ftp? If so, you probably corrupted the file with carriage returns because your ftp defaults to ASCII rather than BINARY. It will be impossible to repair the file because removing all CR characters will likely remove those that were not added as well.

[lawrenzo_1]

Thanks all for your the constructive comments. Although the system is AIX I am presuming that tar works similar across all systems.

The file is corrupt on the system where the file was created.

Chris.

[Steven Schweda]

> [...] I am presuming that tar works similar
> across all systems.

Presuming is often unsafe.

"Similar" is a technical term meaning "not
identical".

This sort of thinking may lead to confused
behavior, such as posting AIX questions in an
HP-UX forum.

In fact, there are many "tar" programs in the
world, each similar to and different from all
the others, in ways which may be important or
not.

[lawrenzo_1]

Hey steve,

if you have a problem with me posting generic unix questions in this forum then please do not respond ....

I use this forum because:

1> I find it the best for a quick and accurate response to most questions
2> People don't usually get hung up that I work on AIX ( I worked on HP for over 4 years )

I can appreciate your comments however what people have posted in this thread has help me come to my conclusion.

[Dennis Handly]

>2> People don't usually get hung up that I work on AIX

If you are using AIX, you must say so in every thread. This will prevent confusion and possibly wrong answers.