EBCDIC to ASCII conversion with dd

[Randy Rayfield]

I am trying to convert a tape (3490) from a IBM mainframe to readable ascii. The block size is 32760 (as seen in the header):

# dd if=/dev/rmt/0n conv=ascii
VOL1LC1779 HDR1S.AMDC
BAF.D021704LC177900010001 0042030050170000000IBM OS/VS 370 HDR2V32760
0204400ZMPKMCDA/STP0217 P B 1912E 0+3 record
s in

I am using the following command:

dd if=/dev/rmt/0n of=./temp ibs=32760 conv=ascii

Once it dumps, it is still in a unreadable fomrat.

I have not done this in 5 or so years, so this info has been purged.....Any help would be appreciated.......

[A. Clay Stephenson]

You are missing the conversion buffer size (cbs=). Typically, a tapeblock contains multiple records (an integral number per block).

[Randy Rayfield]

Would this info be in the header?

[A. Clay Stephenson]

HDR2 Layout:

HDR2, EOF2 or EOV2 label

Bytes Length Example Significance to user
1-3 3 HDR Header label (EOF or EOV possible).
4 1 2 Header label number, 2.
5 1 U Record format. F, U or V (IBM only).
6-10 5 32000 Block length in bytes (maximum).
11-15 5 32000 Record length in bytes (maximum).

16-80 65 spaces

16 1 5 Recording density (IBM). 0-5.
35-36 2 P Compressed data follows (3490 etc.).

-----------------------------------------
Note that 32760 should be considered a maximum block length; the record size is listed as 2044 which is not an integral number of records per block.

I would try this as a test to get the first block of data; if this is variable sized blocks or records then dd is not going to do the job. Also, if this is mixed text and binary data then dd won't work either.

dd if=/dev/rmt/0n of=myfile ibs=32760 cbs=2044 conv=ascii

[A. Clay Stephenson]

Ooops,

Add a count=1 to the dd so that you can do just 1 block initially

dd if=/dev/rmt/0n of=myfile ibs=32760 cbs=2044 conv=ascii count=1

[Randy Rayfield]

I remember something about having to "skip" forward. The command you give does not work, do I need to jump ahead after reading the header?

[A. Clay Stephenson]

Generally there is a header, so you first do a dd if=/dev/rmt/0mn of=header

You don't care about the header so you can send it to /dev/null, if you like. Next there will be 1 or more actual data files that you are interested in. Each of these could have different block/record sizes but the important point is that for each successive file on the tape you use a no-rewind device.

In a few cases, I've found that the header information had absolutely nothing to do with (and in fact, misinformed about) the actual data files that followed. UNIX really has no notion of a tape header or label and sometimes a "one size fits all" header/label is tacked on just to meet some requirement.

[Randy Rayfield]

I can get "data" off the tape, but is is just a hodge podge of control characters. No wonder mainframes are dead.....:-)

[Bill Hassell]

Not a mainfram problem at all. Unix never adopted the standards for tape records. The VOL HDR EOF EOV records made data interchange fairly easy because the details were documented in these header and trailer records. Originally, these were called IBM headers (written in EBCDIC), other mainframe manufacturers (including HP, the MPE operating system) chose ASCII as the character code and the concept of ANSI standard labels became a reality.

However, the author of the tape is probably not aware of data interchange formats and thus created the tape with a convenient (more often: it worked for me one time) JCL not knowing that internal data may need to be converted from binary to character format.

The first thing to do is to get the mainframe author to define the data: is it binary? If so, it is useless outside that mainframe and/or application that created it. Is the data an exported database? If so, what are the fields within the data and what type of data? Ideally, all the data fields will be characters (EBCDIC or ASCII). If this is the case, have the mainframe jockeys dump the headers and the first couple of records on the tape to a printer. Then show the results to the originator of the data to see if they can figure it out.

Mainframes have typically been proprietary (aloof maybe) so data interchange is often a foreign language for the JCL coders. A better choice for data interchnage is "F" rather than "U" or "V". F=fixed records, fixed records per block, ideal for dd to break up. You'll also need to get familiar with the mt command and always use the Berkeley device files to position the tape. Use the command xd -xc to decode the records so they look a lot like mainframe record dumps. To read the headers:

mt -f /dev/rmt/0mnb rewind
dd if=/dev/rmt/0mnb conv=ascii | xd -xc

And to read a couple of data records (the 0mnb device file will not reposition the tape at close):

mt -f /dev/rmt/0mnb rewind
mt -f /dev/rmt/0mnb fsf 1
dd if=/dev/rmt/0mnb bs=2044 cbs=4k count=2 conv=ascii | xd -xc

As Clay mentioned, if the data is text and binary, there is no hope for dd--you'll have to write a decoder program which will require detailed record definitions. (or you can get the JCL guys to perform some convesion for you)