topic Re: Ignite-UX tar errors. in Operating System - HP-UX

Ignite-UX tar errors.

Kevin Boatright — Fri, 30 Jan 2009 13:42:51 GMT

I"m having an issue with a make_tape_recovery on HPUX 11.11. Ignite was working and then stopped.
I originally had 7.7.89 loaded. I have attempted to reinstall ignite and downgraded to an earlier version.

* Creating local directories for configuration files and archive.

======= 01/29/09 09:36:12 EST Started /opt/ignite/bin/make_tape_recovery.
(Thu Jan 29 09:36:12 EST 2009)
@(#)Ignite-UX Revision C.7.7.98
@(#)ignite/net_recovery (opt) Revision:
/branches/IUX_RA0809/ignite/src@75250 Last Modified: 2008-08-25
11:39:14 -0600 (Mon, 25 Aug 2008)

* Testing for necessary pax patch.
* Recovery Archive Description = /dev/rmt/0mn

* Recovery Archive Location = /dev/rmt/0mn

* Number of Archives to Save = 2

* Pax type = tar

In? dsk/vg name minor# Associated disks/mountpoints
2 v /dev/vg00 0x00 /dev/dsk/c2t1d0 /dev/dsk/c2t0d0
/dev/vg00/lvol1 /stand 2
/dev/vg00/lvol2
/dev/vg00/lvol3 / 2
/dev/vg00/lvol4 /tmp 2
/dev/vg00/lvol6 /opt 2
/dev/vg00/lvol7 /usr 2
/dev/vg00/lvol8 /var 2
0 v /dev/vg01 0x01 /dev/dsk/c4t0d1 /dev/dsk/c6t0d1 /dev/dsk/c8t0d1 /dev/dsk/c10t0d1 /dev/dsk/c4t0d2 /dev/dsk/c6t0d2 /dev/dsk/c8t0d2 /dev/dsk/c10t0d2
/dev/vg01/lvol9 /home 0
/dev/vg01/lvol10 /home/emstat 0
0 v /dev/vg02 0x02 /dev/dsk/c4t0d5 /dev/dsk/c6t0d5 /dev/dsk/c8t0d5 /dev/dsk/c10t0d5
/dev/vg02/lvol11 /usr/local 0
/dev/vg02/lvphys /home/phys 0
/dev/vg02/lvconfig /home/config 0
0 v /dev/vg03 0x03 /dev/dsk/c4t0d3 /dev/dsk/c6t0d3 /dev/dsk/c8t0d3 /dev/dsk/c10t0d3 /dev/dsk/c4t0d4 /dev/dsk/c6t0d4 /dev/dsk/c8t0d4 /dev/dsk/c10t0d4
/dev/vg03/lvol12 /or_dump 0
/dev/vg03/lvtables /home/oracle/oradata/PR/tables 0
* Checking Versions of Ignite-UX filesets
* Creating System Configuration.
* /opt/ignite/bin/save_config -f
/var/opt/ignite/recovery/2009-01-29,09:36/system_cfg vg00
* Backing Up Volume Group /dev/vg00
* /usr/sbin/vgcfgbackup /dev/vg00
* Creating Map Files for Volume Group /dev/vg00
* /usr/sbin/vgexport -s -p -m /etc/lvmconf/vg00.mapfile /dev/vg00

* Backing Up Volume Group /dev/vg01
* /usr/sbin/vgcfgbackup /dev/vg01
* Creating Map Files for Volume Group /dev/vg01
* /usr/sbin/vgexport -s -p -m /etc/lvmconf/vg01.mapfile /dev/vg01

* Backing Up Volume Group /dev/vg02
* /usr/sbin/vgcfgbackup /dev/vg02
* Creating Map Files for Volume Group /dev/vg02
* /usr/sbin/vgexport -s -p -m /etc/lvmconf/vg02.mapfile /dev/vg02

* Backing Up Volume Group /dev/vg03
* /usr/sbin/vgcfgbackup /dev/vg03
* Creating Map Files for Volume Group /dev/vg03
* /usr/sbin/vgexport -s -p -m /etc/lvmconf/vg03.mapfile /dev/vg03

* Creating Control Configuration.
* Creating Archive File List
* Creating Archive Configuration

* /opt/ignite/lbin/make_arch_config -c
/var/opt/ignite/recovery/2009-01-29,09:36/archive_cfg -g
/var/opt/ignite/recovery/2009-01-29,09:36/flist -n 2009-01-29,09:36 -r
pa -b 64 -d /dev/rmt/0mn -t 1 -i 1 -m t
* Saving the information about archive to
/var/opt/ignite/recovery/previews
* Creating The Tape Archive
* Checking configuration file for Syntax
tar: Cannot add file -: No such file or directory
tar: Cannot add file 20: No such file or directory
tar: Error exit delayed from previous errors
make_medialif: ERROR: Cannot tar and gzip script files; possibly insufficient disk space in /var/tmp/make_medialif6965.

======= 01/29/09 09:36:48 EST make_tape_recovery completed unsuccessfully

I did not find any files labeled - or 20
There is 5GB of free space in /var.

I was able to tar the filesystems directly to tape without any issue.

I did find this other thread that had the same issue but didn't see a resolution.

http://forums13.itrc.hp.com/service/forums/questionanswer.do?admit=109447627+1233322172759+28353475&threadId=1171617

Any ideas?

Thanks,
Kevin

Re: Ignite-UX tar errors.

Michael Mike Reaser — Fri, 30 Jan 2009 14:21:46 GMT

Somewhere on your system you have a file whose name is of the form

[string-of-characters][:space:]20[:space:][string-of-characters]

IOW, you have a file, somewhere, that has the string " 20 " buried within its name.

I ran into something similar with Ignite several years ago, where Oracle was using Unix-unsociable names eleventeen-gazillion levels deep within their directory structures. After having the DBA either rename 'em or remove 'em (I just don't remember which), tar was happy as a clam.

Re: Ignite-UX tar errors.

Avinash20 — Fri, 30 Jan 2009 14:52:11 GMT

You could find the unprintable character via ls -b

Re: Ignite-UX tar errors.

Steven Schweda — Fri, 30 Jan 2009 15:17:07 GMT

> I did find this other thread [...]

My thoughts are unchanged from back then. I
still find it hard (bordering on impossible)
to believe that any file names are causing
this problem, but if someone can actually
_demonstrate_ a "tar" failure caused by goofy
file names (in the file system, not on the
"tar" command line), instead of merely
conjecturing with no evidence, then I'd be
willing to reconsider.

> [...]
> tar: Cannot add file -: No such file or directory
> [...]

Clearly "tar" is unhappy, but I can't see
the actual "tar" command line, so I can only
guess what bad things might be there. My
_guess_ is that there are some misplaced
options which are being interpreted as file
names, but the weakness of my psychic powers
means that it's only a guess. Who made the
bad command line, and how/why, remain
mysteries.

> [...] but didn't see a resolution.

If you do puzzle it out, be sure to explain.

Re: Ignite-UX tar errors.

Kevin Boatright — Fri, 30 Jan 2009 16:19:38 GMT

Mike, Oracle was installed on this box. Our vendor has been copying files from one box to another. No issues running ignite on the other box. It's running version 5.4.50.

I could not find any files with 20 in between spaces.

I was able to preform:
tar xvf /dev/rmt/0mn on /sbin /dev /stand /usr and /etc without any issues.

I also try using make_tape_recovery -m cpio and still receive the same tar error.

I guess I could try and find Ignite 5.4.50 and install that on the box.

Re: Ignite-UX tar errors.

Bill Hassell — Fri, 30 Jan 2009 18:31:04 GMT

> tar: Cannot add file -: No such file or directory
> tar: Cannot add file 20: No such file or directory

As mentioned, these are bad filenames. Ignite/UX (list_expander) and tar check every filename to be saved. The first is a very common error: The file is just a dash (-), or it is a dash surrounded by non-printing characters. (You can create a file with a dash like this: touch /tmp/-

The second is similar. It can be a file with a 2 followed by 0 but not necessarily together, for example, ESC 2 9 backspace 0 ESC or something similar.

In both cases, you will have to look carefully. You will have to escape the "-" as you search for it, something like this:

find / -name '*-*' -exec ls -dlb {} \;

Note that this will find any file with a dash somewhere. Remove the dash file with a a full path (ie, rm -i /tmp/*-*) Try a similar command for the 2..0 file:

find / -name '*2*0*' -exec ls -dlb {} \;

Re: Ignite-UX tar errors.

sujit kumar singh — Fri, 30 Jan 2009 20:54:18 GMT

hi

also can have a look in addition to what Bill has said at the content of the following:

/var/opt/ignite/recovery/2009-01-29,09:36/flist

regards
Sujit

Re: Ignite-UX tar errors.

Steven Schweda — Fri, 30 Jan 2009 22:37:13 GMT

> As mentioned, these are bad filenames.

Define "bad filename". Perhaps my VMS
experience has biased me, but is HP-UX so
lame that the
official system backup procedure can't cope
with _any_ legal file name? Really??? If
VMS had a problem like that, there'd be a mob
on the move with torches and pitchforks.

Am I the only one in the universe who finds
this explanation hard to credit?

Re: Ignite-UX tar errors.

Steven Schweda — Fri, 30 Jan 2009 22:51:08 GMT

And I know that I'm a bit slow, but can
someone explain why the thing says:

* Testing for necessary pax patch.

and then apparently uses "tar" instead of
"pax":

tar: Cannot add file -: No such file or directory

????

Given the limitations of the typical lame-old
"tar" program (file name length, file size,
who-knows-what-else), shouldn't it _be_ using
"pax", and not "tar"?

What obvious thing am I missing this time?

Re: Ignite-UX tar errors.

Patrick Wallek — Fri, 30 Jan 2009 23:02:25 GMT

pax can write output in either cpio format or tar format. The default is to write out in tar format. So as a result you have many of the same limitations as if you were to use the regular tar command.

If you happen to look at a process listing while a make_tape_recovery is being created then you would see that m_t_r does in fact use pax to create the tape.

Re: Ignite-UX tar errors.

Steven Schweda — Fri, 30 Jan 2009 23:48:23 GMT

> [...] The default is to write out in tar
> format. So as a result you have many of the
> same limitations as if you were to use the
> regular tar command.

Well, there's "tar" format, and then there's
"tar" format. It's been extended repeatedly
over the years, as evidenced by the ability
of GNU "tar" to deal with file names longer
than 100 characters, and files bigger than
8GB. (Eleven octal digits for the file size?
Who writes this stuff?)

> [...] m_t_r does in fact use pax to create
> the tape.

So, unless "pax" claims to be "tar" when it
gets an error, it's hard to explain those
"tar: Cannot add file [..]" complaints,
wouldn't you agree?

My (unfounded) conjecture is that somehow
someone's running "tar" using a "pax" command
line, and the (incompatible) "pax" options
are confusing/annoying "tar". Of course,
that would conflict with the prevailing "bad
file name" theory, but that's how it looks to
me. (But what do I know?)

Re: Ignite-UX tar errors.

Bill Hassell — Sat, 31 Jan 2009 02:58:43 GMT

"bad filenames" is perhaps a misnomer. The Ignite/UX list_expander and the various backup tools are trying to be helpful. There is problem at all in creating obnoxious names with special characters and invisible control characters. The problem is that what you see is not what you get unless you are knowledgeable (how to get rid of "--" or ";" files) and remember to use the -b option for ls as well as enclose spaces inside quotes. Unix isn't unfriendly, it is just indifferent. It follows exactly what you type, and assumes that users know the rules.

Personally I like the list_expander and pax/tar warning messages. It gives me a chance to educate users that create them.

Re: Ignite-UX tar errors.

Steven Schweda — Sat, 31 Jan 2009 03:56:31 GMT

> Personally I like the list_expander and
> pax/tar warning messages. It gives me a
> chance to educate users that create them.

I'm all for education, and if you can get
"tar" (any "tar") to complain about funny
file names which are in the file system but
not on its command line, I'd _love_ to be
educated.

And if you know of a way to persuade
make_tape_recovery to reveal the "pax" (or
"tar") command(s) it generates, _that_ might
be _particularly_ educational.

> [...] the various backup tools are trying
> to be helpful. [...]

Trying but not succeeding? Or are you saying
that it's more helpful to blow up in this
situation than it would be simply to get the
job done and be quiet about it? Perhaps this
is some new meaning for "helpful", with which
I'm unfamiliar.

This whole "bad filename" thing sounds to me
like a bit of folklore with no foundation in
fact. Now, I'm open to a good
counter-demonstration which will show just
how wrong I am, but a bunch of hand-waving
about restrictions which don't seem to exist
will not convince me. Surely, it can't be
very difficult to create a small file system,
populate it with (so-called) good and bad
file names, reproduce this failure, and then
remove the "bad" file names, and see how
splendidly everything works. _That_ would
_really_ be educational.

It would also seem to demonstrate that the
official HP-UX backup software is a steaming
pile of something stinky, but I'm willing to
believe that, too, _if_ someone can actually
demonstrate that it really has this rather
severe problem.

Re: Ignite-UX tar errors.

Patrick Wallek — Sat, 31 Jan 2009 04:26:10 GMT

OK Here's a very simple example.

I created a file called "abc - de f _"

Now, if I do:

$ tar -cvf test.tar a*
abc - de f _

It works fine.

It will even restore
$ tar -tvf test.tar
-rw-r--r-- wallekp/users 0 2009-01-30 22:17:22 abc - de f _

Now the problem comes in when you try to specify the file name on the command line, which pax/tar does with m_t_r:
$ tar -cvf test1.tar abc - de f _
tar: abc: Cannot stat: No such file or directory
tar: -: Cannot stat: No such file or directory
tar: de: Cannot stat: No such file or directory
tar: f: Cannot stat: No such file or directory
tar: _: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors

Do the errors look familiar?

While Unix allows file names with spaces and other character, it can cause issues.

Re: Ignite-UX tar errors.

Steven Schweda — Sat, 31 Jan 2009 04:44:50 GMT

> Now the problem comes in when you try to
> specify the file name on the command line,

Isn't that what I specifically excluded? Why
do you think that I did that?

> which pax/tar does with m_t_r:

It _does_? You know that? Yikes. That
_would_ be a steaming pile. Doesn't it know
about apostrophes? I (yes, even I) do.

> Do the errors look familiar?

Only vaguely. They _are_ different.

I see that "man make_tape_recovery" (11.11)
does mention some restrictions:

[...]
File Names with Non-Printable Characters
Although permissible within HP-UX, it is inadvisable to use characters
that do not have a printable graphic on the hardware you commonly use,
or that are likely to confuse your terminal. Filenames with these
characters cause a warning message to be displayed by
make_tape_recovery. In addition, files that contain these non-
printable characters are not included in the archive.
[...]

I may see the steam beginning to rise...

Of course, I wouldn't consider a space
character to be "non-printable".

I also wouldn't call "make_medialif: ERROR:
Cannot tar and gzip script files; [...]" "a
warning message". But I may be eccentric.

I also haven't yet seen make_tape_recovery
spew the errors (under controlled
conditions), just some elementary shell
globbing problems (which may or may not be
related to what make_tape_recovery actually
does).

Re: Ignite-UX tar errors.

Patrick Wallek — Sat, 31 Jan 2009 04:58:39 GMT

What do you want, Steven? Do you want the developers to stand up a shout "we screwed up"?

File names with spaces, non-printable character etc., can cause problems with tar & pax.

My feeling is that when some of these tools were originally written, the developers never thought about spaces and other things in file names. Yes, there have been updates over the years, but the fact remains that in some cases Unix just doesn't deal well with some things. Is there room for more improvement? Absolutely.

However, I wouldn't call pax, tar, or the Ignite product a steaming pile of just because it doesn't handle some issues well.

If you think that little of it, don't use it. If you think that little of HP-UX, why don't you go back to your precious VMS?

Re: Ignite-UX tar errors.

Steven Schweda — Sat, 31 Jan 2009 05:15:55 GMT

> What do you want, Steven?

What I asked for: a quick demo that
make_tape_recovery really has trouble with
non-trivial file names. (Blame assignment,
always the most important thing, can follow.)

> File names with spaces, non-printable
> character etc., can cause problems with
> tar & pax.

But they don't, really. They _may_ cause
trouble with a _shell_, but that's not the
same thing as causing trouble with "pax" or
"tar". And characters as simple as spaces
don't need to cause trouble for a shell, if
the file names are handled well.

> However, I wouldn't call [...]

A backup program which can't back up legal
files? Boy, that's sure what I'd call it.

> If you think that little of it, don't use
> it. If you think that little of HP-UX, why
> don't you go back to your precious VMS?

I haven't actually left (HP's, not my) VMS.
I just get out a little more than some folks.

I _don't_ use HP-UX much, mostly testing the
Info-ZIP programs, and some other freeware.
(I just hang around in this forum for the
educational value.)

And I didn't say that HP-UX as a whole was
bad, just the defective backup program (if it
really is as defective as people seem to say
that it is).

Re: Ignite-UX tar errors.

Dennis Handly — Sat, 31 Jan 2009 07:33:14 GMT

>tar: Cannot add file -: No such file or directory
>tar: Cannot add file 20: No such file or directory
>tar: Error exit delayed from previous errors

As Steven says, something is wrong here. If ignite uses pax(1) there should be no tar messages. I get:
pax: - : No such file or directory

If you want to see what's really going on, you can use tusc:
tusc -fp -a -o tusc.out make_tape_recovery ...

(Assuming a demon isn't doing the work.)

Re: Ignite-UX tar errors.

Bill Hassell — Sun, 01 Feb 2009 03:16:48 GMT

The list_expander and pax/tar issues don't represent HP-UX filesystem standards. I submitted a bug report for list_expander because it could not handle very large volume groups (hundreds of lvols and pvs). Getting it fixed without introducing a new filename bug was a challenge.

Ignite/UX is a fine product, with the functionality conspicuously missing in most other flavors of Unix. But the choice of a patchwork tar or pax for today's Ignite product needs serious attention. Since Ignite/UX is 100% proprietary to HP-UX, the choice of its file backup tool should be unconstrained and fully compatible with any useable files and filenames (in my not so humble opinion). Ignite/UX tries to prevent saving (and restoring) systems with issues. Perhaps flags can be used to filter some of the warnings. Certainly the error messages can at least show the full pathname.

Re: Ignite-UX tar errors.

Kevin Boatright — Mon, 02 Feb 2009 15:50:13 GMT

I've looked for files that contain any special characters and either a - or 2*0. I have yet to find any.

I did what Dennis suggested and found this in the output.

[11328] execve("/usr/bin/tar", 0x40039d40, 0x40039de8) ... [entry]
argv[0] @ 0x4001ee60: "tar"
argv[1] @ 0x400219d8: "-cfb"
argv[2] @ 0x400219f8: "-"
argv[3] @ 0x40039d68: "20"
argv[4] @ 0x40039d78: "./opt/ignite/data/scripts/os_arch_post_l"
argv[5] @ 0x40039db0: "./opt/ignite/data/scripts/os_arch_post_c"

I'll continue to search.