tar and hidden files

[Dan DeHaan]

I'm running HP-UX 11.0 and using the tar utility to unpackage a file from a vendor. When I extract (or even list) the .tar file I only get a partial extract of the packaged files. When I copy the file to my PC and use WinZip (v 8.1), it is able to see all the files and even extract them. The .tar file was (supposedly) created on a HP-UX server. Is there a problem with my tar utility? Am I missing a secret argument? Am I losing my mind?

[Mel Burslan]

if there is a directpry structure in the tar file, the extraction will contain the same exact structure and you will only see the top level directory information looking form the directory level you started at.

if the tar image has files starting with "." character, i.e., hidden unix files, you will see them using -a switch on ls command, like in ls -al.

Since I am not sure what you mean by hidden files not seen, I am not sure what else to suggest you but if you can provide more information as in, what you can see using winzip that does not show up under tar, you may get better educated responses.

HTH

[Bill Hassell]

HP-UX tar is extremely dumb in that it cannot handle different block sizes automatically. It is common for other Unix flavors to use a different block size, and recent versions of tar can be told to create tar file with a different block size. Normally, tar will abort with a block size error message but not always. Use pax to read the tar package instead. pax handles both tar and cpio and automatically adjusts to different block sizes.

[A. Clay Stephenson]

UNIX has no concept of "hidden files" although those that begin with "." are , by convention, not displayed by ls w/o options. My best guess is that the tar file was created using an unexpected blocking factor. When you say that you get only a partial extract does that mean that files are missing (possibly those that begin with ".") or does it mean that the tar starts and then fails? If so, what error message are you seeing. It is also possible that you are trying to read one of the new, "improved" tar's that supports files up to 8GB or a Gnu tar image which is (almost compatible).

[Raj D.]

Hi Dan ,


# tar -tvf filename.tar
should show all the files on that tar file. There is no hidden file concept for unix with tar.

. files are considered as hidden file though tar can see those ..

Cheers,
Raj.

[Dan DeHaan]

When I run the command 'tar tvf foo.tar' I get a listing of files and directories that are in the foo.tar file. When I look at it with WinZip, I see the same listing, but with 2 extra directory structures that tar did not display. When I use 'tar cvf foo.tar' to extract it, the 2 missing/hidden directories are not extracted, but everything else is.

Can the blocking factor change in the middle of a .tar file? How would you do that?

Here is the output of 'tar tvf foo.tar':
rwxr-xr-x 48/48 0 Aug 19 08:49 2004 netvault/
rw-r--r-- 48/48 316 Aug 19 04:41 2004 netvault/bmsinfo
rwxr-xr-x 48/48 0 Aug 19 04:41 2004 netvault/extrabkl/
rwxr-xr-x 48/48 0 Aug 19 04:41 2004 netvault/extrapackages/
rwxr--r-- 48/48 3959 Aug 19 04:41 2004 netvault/install
rw-r--r-- 48/48 13306 Aug 19 04:41 2004 netvault/license.txt
rw-r--r-- 48/48 22280192 Aug 19 04:41 2004 netvault/nvdist

WinZip is able to display and extract 2 additional directory structures called 'packages' and 'serverpackages'. As you can see, these do not list with the tar utility.

[A. Clay Stephenson]

The blocksize does not change with the tarfile but HP's tar (at least the older versions) only works with 10k (off the top of my head) blocks and ignore the block size in the tar header. Your initial chuck of files fits within that 10k chunk but the remaining blocks are now out of phase. The suggestion to use pax is a good one.

[Dan DeHaan]

Here is the output of the 'pax -vf foo.tar' command:

USTAR format archive
drwxr-xr-x 0 48 48 Aug 19 2004 netvault/
-rw-r--r-- 0 48 48 316 Aug 19 2004 netvault/bmsinfo
drwxr-xr-x 0 48 48 Aug 19 2004 netvault/extrabkl/
drwxr-xr-x 0 48 48 Aug 19 2004 netvault/extrapackages/
-rwxr--r-- 0 48 48 3959 Aug 19 2004 netvault/install
-rw-r--r-- 0 48 48 13306 Aug 19 2004 netvault/license.txt
-rw-r--r-- 0 48 48 22280192 Aug 19 2004 netvault/nvdist
pax: netvault-r2004aug19-chief-hpux.tar : This doesn't look like a tar archive
pax: netvault-r2004aug19-chief-hpux.tar : Skipping to next file...

Could this be a corrupted file?

[A. Clay Stephenson]

I'm beginning to suspect that this was created with a non-standard tar using a blocksize of > 32K (the maximum that pax can handle). If Winzip can extract it, I doubt that the file is corrupt per se but rather non-standard. You could use Winzip to get the files and then ftp them to your box or you might try using the Gnu Version of tar.

http://hpux.connect.org.uk/hppd/hpux/Gnu/tar-1.15.1/

The downside is that since you are 11.0 , you will need to download the source (along with the prerequisites) and compile/link it for youre system. It's not a difficult process and the instructions are included with the download. It becomes more difficult if all you have is the Bundled C compiler.

[Raj D.]

Hi Dan ,

Also you can untar the files in windows , using Winzip , make a ZIP file. FTP to the system.

And unzip using the unzip command.

# cd /usr/local/bin
# unzip -l filename.zip [ To list files ]
# unzip filename.zip

* You need to have unzip file :
Here it is:

http://gatekeep.cs.utah.edu/hppd/cgi-bin/search?package=on&description=on&term=unzip

Cheers,
Raj.

[Raj D.]

Hi Dan ,

You can also , do like this to get the files on unix system, which unix tar unable to retrive:

1. unzip in windows using winzip.
2. make it a single tar file , using:

http://prdownloads.sourceforge.net/gnuwin32/bsdtar-1.02.032.exe?use_mirror=keihanna

The command to be use :

c:\> bsdtar -cvf file.tar yourfiles_path\*



3. ftp the tar file to the unix system.
4. do # tar -xvf in unix system.


Cheers,
Raj.

[Vibhor Kumar Agarwal]

Are the 2 directory structures which are not being listed empty.

Tar sometimes does not extract empty directories.