Operating System - HP-UX
1821246 Members
2757 Online
109632 Solutions
New Discussion юеВ

How to convert ebcdic data?

 
SOLVED
Go to solution
Robert Fisher_1
Frequent Advisor

How to convert ebcdic data?

Hello experts,

I have some legacy data that comes in from an IBM mainframe. All the string data is in ebcdic format and I need to convert it to ascii. My problem is that when I use dd to convert the data all the strings are ok but the binary data is garbage. If I don't use dd, the binary data is perfect but the strings are garbage. Is there anyway to make dd convert only the strings and leave the binary parts of each record alone.

TIA,
Bob
10 REPLIES 10
A. Clay Stephenson
Acclaimed Contributor
Solution

Re: How to convert ebcdic data?

Sorry Bob. That dog won't hunt. Dd is all or nothing in this context. It's time to use C or Perl. You can use the Convert::EBCDIC module from www.perl.org/CPAN to do the conversion and use the unpack() and pack() Perl functions to manipulate the data. You will need to know the exact data layout of each record.
If it ain't broke, I can fix that.
Paul Sperry
Honored Contributor

Re: How to convert ebcdic data?

dd conv=ascii Convert EBCDIC to ASCII.
Elena Leontieva
Esteemed Contributor

Re: How to convert ebcdic data?

Robert,

My company purchased a FilePort for Unix product from SyncSort specifically for this purpose. The product is good and easy to use.

http://www.syncsort.com/products.htm

Elena.
A. Clay Stephenson
Acclaimed Contributor

Re: How to convert ebcdic data?

Actually, if you know just a little Perl this is really, really easy. In your case, you don't need to worry about the individual binary fields because you can simply lump them together - as long as they are adjacent. It makes this conversion quite simple if you know that your binary data imports okay.


If it ain't broke, I can fix that.
Robert Fisher_1
Frequent Advisor

Re: How to convert ebcdic data?

Hi Clay,

I have several files to convert. The simplest one has just a few fields:
Offset Length Description
0 4 Binary integer
4 20 ebcdic string
24 2 Binary integer
26 20 ebcdic string
46 60 ebcdic string
106 48 array of 4 12-byte BCD integers
------
154 bytes total

Can you show me a perl example and then I should be able to handle the other files. Please, please??

TIA,
Bob
A. Clay Stephenson
Acclaimed Contributor

Re: How to convert ebcdic data?

Okay Bob,

This looks like duck soup. Here's my untested (even for syntax) example based on your layout:

--------------------------------------

#!/usr/bin/perl -w

use strict;
use Convert::EBCDIC;

use constant TYPEDEF => 'a4 a20 a2 a20 a60 a48';
# declare each field as binary data, 'a' format follwed by length
# note that we can treat the 4 12-byte BCD's as one field because we
# are not converting them

my $recsz = length(pack(TYPEDEF,()));
my $b;
my $n = read(STDIN,$b,$recsz);
while ($n == $recsz)
{
# unpack fields specified using TYPEDEF in array
my @a = unpack(TYPEDEF,$b);
# convert each EBCDIC string to ASCII $a[0] = 1st field, $a[1] = 2nd, ...
my $s1 = Convert::EBCDIC::ebcdic2ascii($a[1]);
my $s2 = Convert::EBCDIC::ebcdic2ascii($a[3]);
my $s3 = Convert::EBCDIC::ebcdic2ascii($a[4]);
# pack data using unchanged fields from array for binary and
# converted EBCDIC strings where appropriate
my $out = pack(TYPEDEF,$a[0],$s1,$a[2],$s2,$s3,$a[5]);
print $out;
$n = read(STDIN,$b,$recsz);
}
-------------------------------------------

Note that this script functions as a filter, so

convert.pl < oldfile > newfile

Again, you will need to download and install the Convert::EBCDIC module. Installing is easy. After downloading, gunzip it, untar it, and view the README file. It will guide you through the rest of the module install. Again, check this very carefully because I typed in 'on the fly' -- and I ain't exactly known for my good typing.



If it ain't broke, I can fix that.
Elena Leontieva
Esteemed Contributor

Re: How to convert ebcdic data?

Bob,

We used FilePort to migrate from mainframe to UNIX, a lot of data. So, if you need to convert just several files, you do not want to purchase this software, but you probably can get a demo/trial version for free.

Elena.
Robert Fisher_1
Frequent Advisor

Re: How to convert ebcdic data?

Hello,

The perl script actually worked. I just cut and pasted it and ran it. We have been thinking that it would make it easier to process on the HP end if we added a linefeed character to each record. Can we just add a '\n' to the print command? Also, is there some documentation on the pack and unpack procedures?

Thanks,
Bob
Massimo Bianchi
Honored Contributor

Re: How to convert ebcdic data?

Clay is Clay. dot.

[please no points]
A. Clay Stephenson
Acclaimed Contributor

Re: How to convert ebcdic data?

Hi Bob:

I'm glad my advanced one-finger hunt-and-peck typing actually worked. To add a LF, all you need to do is change print $out; to print $out,"\n"; However, THIS IS DUMB. You still need to process it as fixed length records because in your binary data you could easily have a 0x0a (LF) byte that would be perfectly valid data but not be the end of the record.

To get more documentatio, do a man perlfunc. All the standard Perl functions are lumped into this one man page so it's not possible to get jusy the pack and unpack functions. It's a rather big man page so I would print it. viz man perlfunc | lp.

If it ain't broke, I can fix that.