COBOL: Possible odd behavior on file import

[Paul Raulerson]

[With Alpha VMS 8.3, and COBOL 028]

Well, this one surely has me puzzled, so perhaps it will amuse (bemuse?) one of you guys too. :)

I have a fairly large EBCDIC data file to bring over to OpenVMS, and I wasn't sure how OpenVMS would handle EBCIDC, so I brought it down to a Linux system in a binary mode.

The records contant packed fields are fixed length. There are no "record terminator" characters in the file. In other words, except for the packed data, it is about the simplest file you can imagine. 204 bytes per record. The file name is IT02.

I FTP’ed this from the Linux system to the VMS system as a binary file and tried to read it with a definition like this:

SELECT IT02INFILE ASSIGN TO 'RSEN$SRC:IT02.X'
ORGANIZATION IS SEQUENTIAL.
…
FD IT02INFILE.
01 INREC.
05 IN-RECORD PIC X(204).

Had all sorts of trouble. Bugger would read the first record correctly,but shifted every record after that.

Figured it did not like the EBCDIC, so I converted the EBCDIC (except for packed data fields of course) into ASCII and re-FTP'ed it up to the VMS system.

This time I had ASCII and doing a TYPE IT02.X /page on the VMS shows the file in the format I expected to see.

Still had the same problem.

Tried adding a LF after each record (inserted on the Linux system) and reading that, with and without an extra PIC X(1) field in the record. Same problem.

By now I was getting frustrated indeed.

I converted the file to have a CR behind each line, and modified the FD to look like this:

FD IT02INFILE.
01 INREC.
05 IN-RECORD PIC X(204).
05 CR PIC X(1).

Then I accidentally FTP'ed it up in ASCII mode.

Duh.. that worked.

Which leaves me totally puzzled as to WHY it worked, and especially WHY it did not work before I did that. Really totally weird.

I really do NOT want to do CR’s and such, or try to use LINE SEQUENTIAL format, because the data I am reading in has packed fields and other binary information in it. It could conceivably have X’0D’ or an X'0A' or any other binary value contained in the record somewhere.

I’m sort of bemused with this issue and have no idea at all what could be behind it. I initially suspected the FTP program, but it reports that it transfers the correct number of bytes, and FTPing it back to the Linux system results in a file that will binary compare byte for byte with the original.

Also, does the EVE editor have a HEX mode? Or is there a standard hex editor on the system somewhere?

Thanks!


-Paul


(P.S. The COBOL is actually correctly indented and such in the source code, I just didn't force the format here.)

[Hein van den Heuvel · Accepted solution]

First of all, OpenVMS Cobol is the only OpenVMS language which directly supports EBCDIC files.
See its langugage reference manual and the user guide. For example:
http://h71000.www7.hp.com/doc/82final/6297/6297pro_120.html#file_comp_sec

>> so I brought it down to a Linux system in a binary mode.

Every extra step introduces potential problems. Just move it directly to OpenVMS.

>> I FTP ed this from the Linux system to the VMS system as a binary file

You probably ended up with a file which OpenVMS attributes as FIXED LENGHT 512.

If you get the file over like so again, then all you have to do to make is operational should be:

$SET FILE/ATTRI=(MRS=204,LRL=204)

You are fortunte that the record length is even, as VMS rounds up the stored record size to even boundaries, so 203 or 205 would have been trickier, requiring a filler byte in the FD, and a declarations as 204 or 206

>> 05 CR PIC X(1).
>> Duh.. that worked.

Probably with a lot of luck.

In VMS you normally do NOT explicitly make record terminators visible as part of teh record. One lets RMS handle those transparantly, through record attributes


>> Also, does the EVE editor have a HEX mode? Or is there a standard hex editor on the system somewhere?

The most important tools in investigating problems like this on OpenVMS are
1) $DIRECTORY/FULL
Pay close attention to:
Record format: xxx
Record attributes: xxx

2) $DUMP/BLOC and DUMP/RECOR [=COUNT=n]

The DUMP output is what you would expect from a hex editor.

The OpenVMS Freeware has real raw/hex editors. I submitted a simple one as
http://h71000.www7.hp.com/freeware/freeware50/rms_tools/zap.exe

Hope this helps some,
Hein van den Heuvel (at gmail dot com)
HvdH Performance Consulting

[Paul Raulerson]

Absolutely Brilliant!

And I have learned that files under VMS all have more attributes than they do under Unix. I would have found that immediately on a mainframe.:)

I'll figure out the QUOTE command to send to be done with this problem in the future as well.

Thank you!
-Paul

[Steven Schweda]

> Which leaves me totally puzzled as to WHY
> it worked, [...]

Read the FTP RFC. For example:

http://www.w3.org/Protocols/rfc959/
http://www.w3.org/Protocols/rfc959/3_DataTransfer.html

[...]
3.1.1.1. ASCII TYPE

This is the default type and must be accepted by all FTP
implementations. It is intended primarily for the transfer
of text files, except when both hosts would find the EBCDIC
type more convenient.

The sender converts the data from an internal character
representation to the standard 8-bit NVT-ASCII
representation (see the Telnet specification). The receiver
will convert the data from the standard form to his own
internal form.
[...]


Which might be good if you want
EBCDIC-to-ASCII conversion.

Knowing nothing about COBOL, I can't say
much about how to deal with a
fixed-record-length file there, but all that
advice about using DUMP to see what you have
looks good to me.

[Paul Raulerson]

As good as explanation IRT FTP as there is to get. I did understand what the FTP did, just not that it setup a record size for VMS. :)

Thanks -Paul

[Paul Raulerson]

Great answers and I am somewhat less ignorant that 30 minutes before.
:) I really appreciate the answers folks!
-Paul