Apparent libc swprintf bug

[Employee]

I think I've found a bug in HPUX's libc's wprintf and swprintf routines.

I am compiling the following code using GCC 4.0.1 on an HPPA running HPUX 11.11 (11i v1).
I've received the same results decribed below with an old 'Gold' patch cluster installed, with the latest 'Gold' recommended patch cluster installed, and now also with several extra libc patches that I found on hp.com also installed. So it doesn't appear to be a fixed bug.

Here is the code:

#include
#include
#include
#include

#define VALID_DATA L"data"
#define GARBAGE L"\xFFFFFFFF"

int
main(int argc, char ** argv)
{
wchar_t wcs[] = { VALID_DATA GARBAGE L"\0" };
assert(wcslen(wcs) == wcslen(VALID_DATA) + wcslen(GARBAGE));
errno = 0;
if (wprintf(L"%.*ls", wcslen(VALID_DATA), wcs) < 0 || errno) {
perror("wprintf");
return 1;
}
return 0;
}

Ie, I am passing a buffer containing some valid string data followed by some garbage data into 'wprintf', with a format string which should cause it only to print the valid portion of the buffer.

The expected output is the string 'data' being printed to stdout.
The actual output is a message from 'perror' on stderr:
"wprintf: Illegal byte sequence".

If GARBAGE is defined to be something valid such as "more", I get the expected output from my program.
You will note that there are no byte sequences of any sort in my program (I'm not using multibyte strings), and there are also no invalid wide chars in my buffer in the region wprintf should be looking at - but there are invalid wide chars in other regions of the buffer at which wprintf has been told not to look using the precision.

Do people agree with me that this is not the standardised ANSI C behaviour?
I get analogous behaviour from 'swprintf' too.

By experimentation, it appears as if wprintf under HPUX has two seperate bugs:
1. wprintf is performing some sort of validation of the wchar_t values in the string arguments passed to it. This validation is not documented in wprintf(3C). Also, some values > WCHAR_MIN and < WCHAR_MAX do not pass this validation. I am worried about this undocumented apparently-buggy misfeature.
2. wprintf performs this validation up until the null terminator in the passed-in string values, and it does this even if the format string has a precision (as mine does above) which should limit the length of string data inspected. Ie, wprintf incorrectly treats '%.*ls' as '%ls' for the purposes of its internal validation.

As a result of (2), I should be able to crash wprintf by passing it a buffer which is not null-terminated in memory until we over-step our addressable region - even if my format string (supposedly) limits the string length using a precision. It's possible that this could be used for a denial of service attack.

So far I've thought of these workarounds:

1. Add a wrapper around HPUX's 'wprintf'

This would have to scan the format string for %.*ls, and every time it sees it, do a wcsncat instead; but other formats get passed to the system's printf as usual.

Ie, decompose the format string into a list of alternating (safe-for-wprintf, unsafe-for-wprintf) sections, and pass each to either printf if safe, or a replacement for its functionality (wcsncat) if not, in order.

This would make our product significantly slower under HPUX than under other platforms where this extra work isn't required.

2. Port GNU libc's wprintf or someone else's one to HPUX, and use this instead of the HP version.

This is the most attractive option to me at the moment. Does anybody know of an open source or other freely-available implementation of wprintf, swprint etc that we could use? I don't understand whether the LGPL allows this for our commercial app.

3. Arrange for all strings passed to wprintf to be null-terminated, even if their corresponding formats have precisions that mean that null termination shouldn't be required.

This would require significant changes to our product source, and thus significant development effort, and it's still ugly.

Can anybody think of any other workarounds?

How do I report defects like this to HP? We do not have a support contract with them.
Do HP traditionally fix defects of this sort? If so, how quickly do they do so?

[Sreedhar Nathani]

Hello Employee,

Thanks for proving such detailed message. We passed the infomration to the HP labs for verification. For your reference please find the SR Number 8606411049 which logged for this problem

Thanks