bug in swprintf

[Thomas Prause]

There seems to be a bug in the swprintf() function on Tru64 5.1. The following lines of code:
rc = swprintf(wseq, sizeof(wseq), L">%4d %12S %12S %5ld %S<", 1, L"2nd", L"3rd", 444, L"last");
printf("%S rc = %i\n", wseq, rc);
rc = swprintf(wseq, sizeof(wseq), L">%4d %12S %5ld %S<" , 1, L"2nd", 333, L"last");
printf("%S rc = %i\n", wseq, rc);

produce the following output:
> 1 2nd 3rd 444 rc = 32
> 1 2nd 333 last< rc = 30

but I would expect:
> 1 2nd 3rd 444 last< rc = 43
> 1 2nd 333 last< rc = 30

which is what it prints on several other platforms. Looks like the first line gets tuncated on 5.1. On 4.0 its even worse. The wide character strings are not printed:
> 1 444 < rc = 39
> 1 333 < rc = 26

This all has been tested on the HP testdrive maschines, which should be up2date.

My questions:
1. Is there anything wrong with the sample code?
2. If not, where can I report this WITHOUT having a support contract?

[George Smithson]

Hi Thomas,

I've just discovered these two apparent bugs on a Tru64 4.0F system. wprintf is broken as well as swprintf. I believe the 4.0F system has the latest patch bundle installed (not certain). I can't find any mention of these bugs being fixed in the release notes for the latest patch bundle. Digging around Google finds nothing useful apart from your post, and digging in hp.com for 'patch libc', 'printf', etc didn't find anything relevant.

I did find patches for a Tru64 problem regarding wprintf interpreting its args as byte arrays not wchar arrays when it sees '%S'. But that doesn't seem to be the problem here - if it were, wprintf(L"foo") would not work, and that does work.

I've discovered that under Tru64 4.0F, the second bug (wide strings aren't printed) can be worked around as follows. Given:
wchar_t * s = L"Hello, world!\n";
...then this code incorrectly outputs nothing:
wprintf(L"%ls", s);
...whereas this code works correctly:
wprintf(L"%.*ls", wcslen(s), s);

The behaviour of these two calls should be identical.
The same workaround works for swprintf too.

So it seems as if something inside [s]wprintf incorrectly calculates the length of the wide string. But wcslen can correctly calculate the length of the wide string. So perhaps Tru64's [s]wprintf contains its very own broken re-implementation of wcslen.

I also see on 4.0F the first bug you mention, where the last argument is not printed. Actually, it's even worse for me: a seemingly arbitrary amount of data at the end of the format string is ignored. It doesn't seem to matter whether that data is format specifiers such as %ls, or literal text. I can't think of a way to work around this problem.

This behaviour occurs both when I compile using GCC 4.0.0, and when I use the system 'cc' which calls itself "DEC C V5.9-011".

I think what I will do is make sure the latest patch bundle is installed, and if that doesn't fix the problem, then I'll look for a public-domain reimplementation of these broken functions, and link that into our software on Tru64, and if I can't do that, then we'll just advise customers who need versions of our software built with wide character support that they need to switch to Linux. From what I hear Linux runs really well on Alpha...

BTW, to be a little nit-picky, there are two very minor problems with your code. But neither of which could cause the problems you're experiencing. But since you asked:

1. Passing sizeof(wseq) as arg 2 to swprintf

I think you mean sizeof(wseq) / sizeof(wseq[0]), because arg 2 to swprintf counts wide chars, not bytes.
Using sizeof at all is only correct if wseq is an array. If wseq is a pointer, sizeof(wseq) will return 8 (on the 64-bit Alpha) and your results would be incorrect.

2. Using non-ISO C '%S'

The '%S' format is a non-standard extension to wprintf. The standardised ISO C equivalent is '%ls'.

I hope this helps somehow. If I find out anything useful further, I'll post here again.

[George Smithson]

I just discovered that the latest patch kit from HP for Tru64 V4.0F is already installed on our Alpha/Tru64 box. This patch kit comes in a file named 'DUV40FB22AS0008-20030730.tar' which I downloaded from HP's web site. So it looks like HP don't have a fix for this problem.

Of course, it could be that this functionality originally worked after base operating system install, but was later broken by installation of one of these patches... I haven't tried backing out patches or reinstalling Tru64 (yet).

HTH

[Thomas Prause]

Don't waste your time reproducing this with older patch levels. As far I can remeber we could reproduce this on a quite fresh installed system too.

[George Smithson]

OK, I've proven what I thought, that this is a bug in Tru64's libc.

I investigated this problem on another Alpha/Tru64 box ('theirs') in addition to our Alpha/Tru64 box ('ours'), using both static and dynamic linking. Both boxes are 4.0F; ours has the latest patch bundle, and theirs appears to have it too. I got the following results:

Compile host Compile type Run host Result
ours dynamic ours fail
ours dynamic theirs pass
ours static ours fail
ours static theirs fail
theirs dynamic ours fail
theirs dynamic theirs pass
theirs static ours pass
theirs static theirs pass

This strongly implies it is a libc bug.

Then I grabbed a copy of /usr/ccs/lib/libc.a (the static version of the C runtime library) from their box, and recompiled my test program on our box, but this time with GCC's '-nostdlib' option (which causes GCC not to link with the standard runtime libraries etc). I explicitly specified the filename of the other machine's libc.a file on GCC's command line. Ie, the only thing I am changing is that I am now using the other machine's libc - but I am using the same compile host, compiler, source, everything. This resulted in a working binary. So it is certainly a libc bug.

It's still a bit odd on several counts:

1. Their system has a significantly older version of libc.a than ours. How come we have different versions? Both machines have the same patch set installed (4.0F's latest), I think.
2. How come the newer libc is buggier than the older one? One would think/hope that libc gets less not more buggy as time goes on...
3. How come you and I are the only people Google can find who are hitting this problem? This is a show-stopper bug in basic functionality; why isn't the web full of people complaining about this? The bug must have quite a small envelope, which you and I are unfortunately in. I wonder what exactly that envelope is.

I think what I will do now is replace the apparently-broken wprintf.o inside our libc.a with the apparently-working wprintf.o from inside their libc.a, and see whether the resulting mixed-version libc.a:
1. Works (doing this could cause crashes if the internal calling conventions/parameters have changed); and
2. Fixes the wprintf bug.

If that doesn't work then I think I will replace our entire libc.a with theirs. Obviously that's rather drastic.

I suppose I could email to you a working wprintf.o or libc.a if you want. I suspect that's a copyright breach though...

Good luck!

[George Smithson]

OK, I moved the existing /usr/ccs/lib/lib.a and libc_r.a to files with different names, then created copies of them but using the 'wprintf.o' member out of the libc.a and libc_r.a (respectively) from the other Tru64 box instead of the existing one, then installed these new, hacked versions under the normal names in /usr/ccs/lib.

The useful commands are:
#/bin/ar -x libc.a
#/bin/ar -r ours/libc.a theirs/wprintf.o

I tried this with GNU 'ar' first, but it made archives that caused Tru64's ld to crash :-(

Programs that call wprintf and are linked statically are working on our Tru64 box now. Dynamic-link wprintf binaries still don't work, and I don't have a solution for this.

Can .so dynamic libraries be updated by replacing one object file included within them (wprintf.o) in the same way that .a static archive library can be? I believe not.

But working static binaries are good enough for us, so I am happy and am now dropping this issue.

It's a bit unimpressive that this bug is present in Tru64, but defects do occur in software and I accept that.

But it is unforgiveable that this defect has not yet been corrected in Tru64's libc. Tru64 libc engineers, fix your code and get a patch released!

Good luck Thomas.

[Ann Majeske]

Have you tried this on the most up to date version of Tru64 UNIX with the most recent patch kit (V5.1B with Patch Kit 5)? V4.0F and V5.1 are both very old versions that are no longer supported. There hasn't been a new patch kit on either one for 2 years.

[George Smithson]

Ann,

We do not need our software to work once on one given Alpha. We need it to work always on any given Alpha.

It doesn't matter to me whether or not it works with newer versions of Tru64. It needs to work on every version of Tru64, because we have customers on pretty much every version of Tru64. Even if I could prove that the bug did not exist on Tru64 version X, what am I supposed to do? Try to convince all of our customers using Tru64 v
We need a working Tru64 libc.a that we can link statically against. Now we have one (no thanks to HP).

I am really unimpressed by this defect. The sooner Tru64, HP-UX etc etc die and are replaced by Linux the better, in my opinion. It's been a long time since I remember a glibc bug of this severity; and glibc fixes take much less than two years to come out.
And given libc source, I could have easily fixed this bug in less time than it has taken to write these forum posts.

Thanks anyway.