Operating System - OpenVMS

Re: CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

 
Steven Schweda
Honored Contributor

CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

Here's an interesting quirk into which I
recently ran. I was trying to build a recent
OpenSSL kit (1.0.0d) using 64-bit pointers.
(CC /POINTER_SIZE=64=ARGV [...].)

After getting past most of the easy problems,
I ran into some test failures whose cause was
unexpected, perhaps even novel. Several of
the [.apps]*.c programs scan their
command-line argv[] array using code like
this:

[...]
int MAIN(int argc, char **argv)
[...]
char **args;
[...]
args = argv + 1;
[...]
while (*args) [or similar]
[...]
args++;
[...]

That is, they look for a NULL terminator at
(past) the end of "argv". (I might have
expected to see this sort of thing with
envp[], but I'd've figured that argc was there
for a reason, and I'd've used that for
determining where argv[] ended. But it's not
my code.)

This NULL-terminator expectation seems to work
well enough on VAX and IA64, and on Alpha with
32-bit pointers, but, on Alpha with 64-bit
pointers, it breaks down. In that case, these
programs frequently see one extra garbage
argument on their command lines, leading to
symptoms like:

Error opening recipient certificate file ÀØÜz

where "ÀØÜz" is what appears to follow the
usual (real) command-line arguments in argv[].

The obvious fix is to re-write these programs
so that they use argc, and don't look for a
NULL terminator at the end of argv[]. But,
it's apparent that some people seem to expect
this thing to be NULL-terminated, and in this
situation, it's not. For all I know, there's
some POSIX standard (or something) which
demands that argv[] be NULL-terminated, in
which case, this would seem to be a bug. In
any case, there would seem to be a plausible
argument to be made for making this work as
well on Alpha as it seems to on IA64 (whether
that's from good design or just good luck).
(And if it is only good luck, then it might be
good to add a little reinforcement to the
luck.)

alp $ cc /version
HP C V7.3-009 on OpenVMS Alpha V8.3


With no patch access these days, this is all
of only academic interest to me, but I thought
that it might be of some interest to a broader
audience.
16 REPLIES 16
John Gillings
Honored Contributor

Re: CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

Steven,

Interesting. I'm curious to know if this is one of those things that fell out of someone's implementation, that are now widespread enough to warrant standardising.

Since your pointers are now 64 bit, is it possible there's a 32 bit null, followed by junk?

If I've translated your magic string "Ã Ã Ã z" correctly, it's hex 7ADCD8C0, which on my V8.3 system appears to be inside DCL. Does argv always map to the same location? (I don't have a C compiler to experiment with).
A crucible of informative mistakes
Craig A Berry
Honored Contributor

Re: CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

The standard at

http://c0x.coding-guidelines.com/5.1.2.2.1.html#170

or 5.1.2.2.1 of C99 if you're following along elsewhere, says

"â argv[argc] shall be a null pointer."

So I think you've found a bug.
Craig A Berry
Honored Contributor

Re: CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

Also, the POSIX specs on exec and friends

http://pubs.opengroup.org/onlinepubs/009695399/functions/execve.html

say, "The argument argv is an array of character pointers to null-terminated strings. The application shall ensure that the last member of this array is a null pointer. "
Steven Schweda
Honored Contributor

Re: CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

> If I've translated your magic string "Ã Ã Ã z"
> correctly, [...]

That's not really the pointer itself, but the
"string" it pointed to, although it's not far
off. My fprintf() diagnostic output looked
like this:

[...]
cms. argc = 10.
[Here, it's looking at the non-option
(file-name) arguments. "argc_x" was my
temporary count (-down-to-one) of valid
remaining arguments.]
cms(1). argc_x = 3, *args: >smime-certs/smrsa1.pem<.
file_ctrl(). ptr: %x00000000004e077a >smime-certs/smrsa1.pem<.
cms(1). argc_x = 2, *args: >smime-certs/smrsa2.pem<.
file_ctrl(). ptr: %x00000000004e0791 >smime-certs/smrsa2.pem<.
cms(1). argc_x = 1, *args: >smime-certs/smrsa3.pem<.
file_ctrl(). ptr: %x00000000004e07a8 >smime-certs/smrsa3.pem<.
cms(1). argc_x = 0, *args: >Ã Ã Ã z<.
file_ctrl(). ptr: %x000000007adcd870 >Ã Ã Ã z<.
Error opening recipient certificate file à à à z
[...]

("file_ctrl()" is the function which actually
tried to open the file by name.)

So, unless I bungled something, that last
"ptr:" value (which looks significantly
different from the earlier, valid ones) is
what everyone was hoping would be all zeros.


> [...] Does argv always map to the same
> location? (I don't have a C compiler to
> experiment with).

I believe that it was pretty reliable, but
once I found out what was happening, I
stopped paying close attention to the
details. I figured that someone must know
who builds argv[], and how, so if this was an
actual bug, then it should be pretty easy to
investigate and/or fix.

> [...] (I don't have a C compiler to
> experiment with).

That's a pretty shameful admission, isn't it.


> The standard at [...]

Hmmm. Those certainly are suggestive.
Perhaps it was always thus, and I, blinded by
the existence of argc, never noticed. Or,
perhaps whoever's setting this stuff up
thinks that it's being done properly, but
apparently it's not. Certainly, most of the
code I've seen looks at argc, not for a NULL
at the and of argv[], and most VMS users
don't compile most stuff with a 64-bit
argv[], so a problem like this might easily
have been missed.
Craig A Berry
Honored Contributor

Re: CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

Here's a simple reproducer. Works ok on Itanium, but fails on Alpha (though not a particularly up-to-date on ECOs Alpha). Hard to say that it's a compiler or CRTL problem for sure since something might have already gone wrong before language support took over from the image activator.

$ create argv_of_argc.c
#include
#include

int main(int argc, char *argv[])
{
void *p = argv[argc];

if (p != NULL)
printf("not ");

printf("OK - argv[argc] should be NULL (got %p)\n", p);
}
^Z
$ cc/pointer_size=32 argv_of_argc
$ link argv_of_argc
$ r argv_of_argc
OK - argv[argc] should be NULL (got 0)
$ cc/pointer_size=64=argv argv_of_argc
$ link argv_of_argc
$ r argv_of_argc
not OK - argv[argc] should be NULL (got 7FF57810)
$ cc/vers
HP C V7.3-009 on OpenVMS Alpha V8.3
John Gillings
Honored Contributor

Re: CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

>That's a pretty shameful admission,
>isn't it.

au contraire!

Having spent almost 20 years as a tech support "linguist", dealing with all manner problems reported in the full spectrum of programming languages, C was, by far, the worst for programmers reporting "I've found a bug" when the real issue was they didn't understand what they'd really asked the compiler to do (next in line was Basic, but that's another story)

Far too heavy on obscure syntax "swiss army knife" control structures, and far too light on easily understood data structure definitions.

Indeed, this report is a perfect example of the fundamental problem with the language. Proponents of C would laud the "freedom" to code something as basic as processing an argument list in different ways.

I'd expect a programmer to be able to glance at and immediately recognise argument parsing without having to figure out which variation the programmer has chosen.

The dependence that a programmer is supposed to "just know" that an argument list is terminated by a null, along with the volumes of other critically important language-lawyer trivia is a recipe for building in bugs.

All programming languages have ideosyncratic faults, but C takes the art form to a whole new level. The only reason the language survives at all is the strategic history of giving compilers away free to universities in the late 1970's, and the resulting momentum from programmers who never knew any better. (At the same time a BLISS compiler license was US$20K per year)
A crucible of informative mistakes
Lucifer Megacruel
Valued Contributor

Re: CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

Hi Steve,

I am not trying to reduce the severity of the problem. But a more POSIX ey way of doing it these days I guess is to use the getopt family of functions than messing with the command line options in the application anyway :). Can you just see if that works ?

--Lucifer

"To Denouce the Evils of Truth and Love. To cause may hem and destruction and necromancy , Lucifer is here"
Steven Schweda
Honored Contributor

Re: CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

> [...] Can you just see if that works ?

Perhaps I could, but I won't. It's not my
code, and I really doubt that the OpenSSL
folks would like to drag all the getopt stuff
into their product, even if it would help
work around this (VMS-specific) problem. (I
know that _I_ wouldn't.)

> [...] a more POSIX ey way [...]

The _most_ POSIX-y way of doing anything is
usually to make the C infrastructure
(compiler, RTL, ...) follow the POSIX
standard, which seems not to be happening
here.

My initial work-around was to re-write the
command-line processing code to look at argc,
but, as the head OpenSSL guy suggested, it's
a smaller change to check
if (argv[ argc] != NULL)
in a central location (which exists), and
then, if necessary, make a properly
terminated duplicate argv[], and then pass
that along to the consumers, instead of the
original, defective argv[]. It's ugly, but
it works (and it's more compact than "fixing"
the several consumers).

I sent a short message to
OpenVMS.Programs@you-know-where, citing this
thread, so I'm sure that the appropriate
engineers will soon be all over this problem.
Perhaps someday I'll even be able to get hold
of the patch which fixes it, but I'm not
holding my breath.


> au contraire!
> [...]

With great power comes great responsibility.

> [...] a programmer is supposed to "just
> know" [...]

Or just read one of those standards
documents. Of course, it helps if the
underlying implementation is bug-free.

> [...] The only reason the language survives
> at all [...]

There's a reason that people try to gain
market share for a new product. Momentum,
once gained, can require significant force
and/or time to extinguish. (Ask Newton.)
Lucifer Megacruel
Valued Contributor

Re: CC /POINTER_SIZE=64=ARGV, Alpha, NULL termination of argv[]?

Hi Steve,

The _most_ POSIX-y way of doing anything is
usually to make the C infrastructure
(compiler, RTL, ...) follow the POSIX
standard, which seems not to be happening
here.


I am not denying that, let me take a look at it deeper, may be when i have some more time.

--Lucifer

"To Denouce the Evils of Truth and Love. To cause may hem and destruction and necromancy , Lucifer is here"