Limitations of binary mode and stream files

[Ben Armstrong]

While working on the OpenVMS port of Ruby, I'm running into an apparent C runtime limitation of binary mode.

A small ruby test illustrates the binary output problem:

File.open('temp.tmp','wb') {|output| output.print "ABC\xFFDEF"}
$ dump temp.tmp
Dump of file DSA0:[DM]TEMP.TMP;1 on 14-JUN-2005 11:05:43.87
File ID (201050,63,0) End of file block 1 / Allocated 69
Virtual block number 1 (00000001), 512 (0200) bytes
00000000 00000000 00000000 00000000 00000000 00000000 00000000 FF434241 ABC............................. 000000
...

A C++ port of this test shows that this is not just Ruby's fault:

#include
#include
int main()
{
ofstream testf("test.dat");
if(testf) {
testf << "ABC\0xFF DEF" << endl;
}
testf.close();
return 0;
}

This produces the same output as above. That is, the "\0xFF" output to the stream file is considered an EOF, and the remaining output is discarded.

Simply switching to a fixed, 512 byte record format is not an acceptable solution because then we lose the ability to write binary files that are a precise (non-512-multiple) number of bytes long.

So, how do I get there from here? How do I write binary files of arbitrary lengths?

Ben

[Antoniov.]

Ben,
C ANSI introduced on 1995 a new rule:
after backslash and x, programmer has to declare hex value with 4 characters to avoid confusion like your example: \xFFDEF where FFDE is a valid hex value.
The right declaration is \x00FFDEF.
I don't know if this is the troubel trouble, but you can quickly prove.

Good luck.
Antonio Vigliotti

[Bojan Nemec]

Ben,

The problem in yours C++ example code is:

testf << "ABC\0xFF DEF" << endl;

instead of

testf << "ABC\xFF DEF" << endl;

Note the missing 0 in the second line. In yours code the \0 is translated as a null character and the string is terminated at this point.

Bojan