read/write binary file with HP C

[Jeff Shangguan]

Hi all,
I'm a new C programmer on VMS.
I find C programing is not the same as on Windows. for example, I want to read a TEXT file in BINARY mode, and then output the content in a new file. So I write the following code segment:
const int maxSize = 1024;
char buffer[maxSize];
FILE *fp = fopen(inFilename, "rb");
int size = fread(buffer, 1, maxSize, fp);
fclose(fp);
fp = fopen(outFilename, "wb");
fwrite(buffer, 1, size, fp);
fclose(fp);

but after running this code segment, I find there's no newline character in the output file, say, all the contents in the input file are merged in one line.

I think the problem may be the rfm or ctx option. I tried to set these options but still no correct output.

So I wander how to duplicate a file exactly as the original one.
Can any one help me on this issue?
BTW, the rfm of the original input file is VAR.

Thank you

[Hoff]

Forget what you know about Windows when you're working on OpenVMS; you're in a whole different world.

As for your code, I expect the newly created file is marked as a binary file. I'd have to look at a DIRECTORY /FULL and a DUMP of the file contents to see what was in it. But if the file was created binary, I'd expect OpenVMS to display it as binary data. Then there's the "fun" of line termination. Windows doesn't "do" this sort of low-level sequential file format stuff, and OpenVMS has multiple line terminators for sequential files available to the programmer.

Sequential files come in a variety of formats. To see which sequential formats are in use here, use the DIRECTORY /FULL command.

Another way things are different: you're using a Unix emulation layer within the OpenVMS C to get to to a native operation; the native I/O APIs are built on RMS, and XQP-level calls and $qio and $io_perform system service calls. The C stuff is layered on these, and the design goal here is Unix compatibility. (Which means not only are you not on Windows any more, you're kind-of-sort-of on Unix here, which can really throw a brain cramp. It did for me, certainly.)

Best way to duplicate a file? lib$spawn() or system() a COPY command, or use callable convert conv$conv() (yes, convert, unfortunately there is no callable copy) in the OpenVMS libraries, and the C standard library lacks an API for same.

If you poke around the network, you'll find examples such as Carl's Callable Copy

http://labs.hoffmanlabs.com/node/417

and you can Google for an example program called "FAST_IO_COPY.C" which uses newer APIs and RMS to do a file copy operation. (That's a bit fancier than Carl's stuff.)

The record format (RFM) is in play here, yes. See the RMS manual for the definitions of that. And for better control of the file you are creating, have a look at fdl$create():

http://labs.hoffmanlabs.com/node/595

You'll definitely need to read the C manuals in the C shelf, and (over in the OpenVMS shelf) the Programming Concepts Manual, and the Guide to OpenVMS File Applications manual and (yes, in the OpenVMS shelf) the HP C run-time manual, if you've not already done so. You'll find yourself referring to the RMS Services Manual (that's a pretty dense manual), too.

And again, you'll be better served by forgetting what you know about Windows or whatever other platforms you might know; that path leads to frustration when working on a new (and vastly different) OS platform.

Stephen Hoffman
HoffmanLabs LLC

[Hein van den Heuvel]

To me, this C-RTL behaviour is a bug.
You told it 'binary'... make it be so.
If I recall correctly the problem is on the input side where various attributes tell the CRTL to try to 'HELP' and it switches to RECORD mode, after that it applies the BIN as in no record terminator handling, but it is too late. The record data is read, not the block data.

You can sort of work around it by $ SET FILE/ATTR=RFM=UDF, or FIX
But you don't really want that.

Note, the feature logical DECC$DEFAULT_UDF_RECORD does NOT help here either. :-(.

I've pretty much given up trying to battle CRTL from trying to 'help' me and switch to native RMS calls almost always depending on what I really want to accomplish.

If you want to copy a file.. you may want to spawn (system) a copy command.

So... what problem are your really trying to solve?


If you do stick to fopen, beware of its documented behaviour:

If a version of the file exists, a new file created with fopen inherits certain
attributes from the existing file unless those attributes are specified in the fopen
call. The following attributes are inherited:
Record format
Maximum record size
Carriage control
File protection

Hein.

[JagadishM]

Hello,

The current behavior is as documented in the CRTL reference manual:
=========================================================================
1.8.2.2.1 Accessing Variable-Length or VFC Record Files in Record Mode

When you access a variable-length or VFC record file in record mode, many I/O
functions behave differently than they would if they were being used with stream
mode. This section describes these differences.

In general, the new-line character (\n) is the record separator for all record
modes. On output, when a new-line character is encountered, a record is
generated unless you specify an optional argument (such as "ctx=bin" or
"ctx=xplct") that affects the interpretation of new lines.
=========================================================================

Thank you,
CRTL Team.

[Hein van den Heuvel · Accepted solution]

JagadishM,

I for one, as a bystander, much appreciate the "CRTL Team" getting involved. It's good to get confirmation that we are not just talking to ourselves. Thank you!

>> The current behavior is as documented in the CRTL reference manual

Sure, but that's not the problem.
The problem as I see it is more simple.
I hope my deescription matches Jeff's

Imagine a STREAM-LF file with 3 lines:
aap
noot
mies

Now I want to read in BINARY mode and see _my_ (RMS injected) linefeeds, not some C-RTL generated helper which looks like a line-feed.

If I read in binary mode, NOT record mode, the read returns:
aapnootmies.

That does not match what I expect from binary. That tells me that the CRTL looked at the records and interpreted them, opting to drop the RMS line terminators (be they linefeeds or the RMS variable length control bytes.

That just does not seem right to me.

Please help us understand how to use the CRTL to read the raw bytes from the file.
In my case notably the RMS record length word and alignment byte for variable length files.
Jeff is probably happy just seeing the LF or CR-LF.

Trivial test program below.
Using the debugger you'll see that the problem happens on the READ side, as buffer gets loaded with "aapnootmies" in my example.

Cheers,
Hein.


#include
main (){

const int maxSize = 100;
char buffer[maxSize];
FILE *fp = fopen("in.tmp", "rb", "ctx=bin");
int size = fread(buffer, 1, maxSize, fp);
fclose(fp);
fp = fopen("out.tmp", "wb");
fwrite(buffer, 1, size, fp);
fclose(fp);
}

[John Gillings]

>If you want to copy a file.. you may want
>to spawn (system) a copy command.

A long standing hole in OpenVMS RTLs there is no callable COPY. Next best is callable CONVERT. See CONV$CONVERT.

[Hoff]

The C RTL is good at Unix I/O emulation. If I'm hauling Unix code around, I use it. If I'm looking to use RMS-native formats (sequential, indexed, etc), then I tend to use direct RMS calls.

Better control is certainly possible through the RMS keywords available on the C I/O calls (an OpenVMS platform extension), but I tend to go directly to RMS.

Here is a BSD license C library for calling directly into RMS on OpenVMS:

http://labs.hoffmanlabs.com/node/1260

And here are some related programming notes on using C on OpenVMS:

http://labs.hoffmanlabs.com/node/273

[Jeff Shangguan]

Thank you all for your reply.
Actually, what I really want to is to develop a tool which can do binary replacement for files, e.g. replacing all "0x0A 0x0B 0x0C 0x0D" with "0x1A 0x2B" in a binary file, under VMS.

I tried to "set file/attr=rfm=udf" for the input file, with the following content:
aa
bb
cc
And "fwrite" it to another file. The most strange thing is that, SOMETIMES the content of the output file is correct, with newline and return characters. BUT sometimes, even I exactly repeated the same procedure, the result is without newline and return. I tried for many times, still with this problem

[Hoff]

>Actually, what I really want to is to develop a tool which can do binary replacement for files, e.g. replacing all "0x0A 0x0B 0x0C 0x0D" with "0x1A 0x2B" in a binary file, under VMS.

Ah, ok. You're rewriting some combination of xxd and patch and zap tools. Cool.

http://labs.hoffmanlabs.com/node/940

[JagadishM]

Hello Jeff,

To have a similar behavior as on Windows, we would require a Feature Request.

Please raise a Feature request for OpenVMS CRTL.

Once we have the feature request, we will plan for the future releases.

Thank you,
CRTL Team.