file conversion using dd

[steven Burgess_2]

Hi all

I have a file that has the following block size and record length

REMOTE_SYSTEM_TYPE=MVS
RECORD_LENGTH=32756
BLOCK_SIZE=32760

The file has come from remote VMS and looks as though it has been blocked for space purposes.

It comes through as awk program text as the HP doesn't know what type of file it is

PROD.tsysuk-edouk-stmts.20030307231003: awk program text

I am trying to convert the file to ascii using dd with the following permitations

dd if=PROD.tsysuk-edouk-stmts.20030307231003 conv=ascii bs=32760 of=ste.new

dd if=PROD.tsysuk-edouk-stmts.20030307231003 conv=ascii cbs=32760 of=ste.new

dd if=PROD.tsysuk-edouk-stmts.20030307231003 conv=ascii bs=110 of=ste.new

dd if=PROD.tsysuk-edouk-stmts.20030307231003 conv=ascii cbs=110 of=ste.new

Plus others. The cbs=110 is giving me some readable text but not enough.

Can anyone advise further ?

Thanks in advance

Steve

[melvyn burnard]

Steven
try your first attempt again, but use ibs=32760 as opposed to bs=32760

More often than not, this is a long drawn out play with all hte variables until you get what you want out.

[steven Burgess_2]

Hi

Struggling -

Steve

[melvyn burnard]

ok, so now try:
dd if=PROD.tsysuk-edouk-stmts.20030307231003 conv=ascii ibs=32760 cbs=32756 of=ste.new

[steven Burgess_2]

Hi Melvyn

Some ascii char but the rest is just control characters

Thanks

Steve

[melvyn burnard]

Ouch, this may be a painful process. Hopefully there may be others in the forum who might be able to shed more light on this for you.

[Jean-Louis Phelix]

Hi Steve,

A long time ago I had to play this kind of game ... But I don't known VMS. Are you sure source coding is EBCDIC ? Sometimes I also had to play with conv=swab (to swap bytes) or conv=unblock to convert from fixed-length to variable-length.

Regards.

[H.Merijn Brand (procura]

MVS != VMS :)

What is it you want to achieve? Read blocks of 32760 bytes, strip trailing spaces and write to (ASCII) file with newlines instead?

can you make the first 100k of the file available as URL, so we can have a look? I bet it's doable with perl, whatever you want to achieve :]

Enjoy, have FUN! H.Merijn

[Ian Box]

I'm guessing here, but if the blocksize is 32760 and the record length is 32756 then you've probably got what the IBM world calls variable blocked records.

If this is the case, then the first 2 bytes of each block will hold the binary length of the block, and then each record within the block will have a 2 byte binary record length at the beginning. In each case the third and fourth bytes should be zero (they are used for something, but it's so long since I did anything with these records that I can't remember what). Each record can be up to 32756 bytes, but could be much shorter.

Sorry, I don't know if there is any standard utility on HP-UX which will re-block them.

How did you get the data? Was it ftp'd from an MVS mainframe? If so there should be some options on the ftp transfer that might allow reblocking. A 'help site' in ftp might provide assistance.

HTH

Ian

[H.Merijn Brand (procura]

If Ian is right, try this:

#perl -le'while(read STDIN,$_,32760){print unpack"s/a",$_}' file.vms >file.txt

variations of s/a to n/a or v/a (big- vs. little-endian numbers) could help

Enjoy, have FUN! H.Merijn

[steven Burgess_2]

Hi everyone

Thanks for all your input. I have had a really busy morning , will try out your suggestions this afternoon and get back to you

Thanks again

Steve

[steven Burgess_2]

Hi

And yes - it's from mvs , not vms (doh!) and variable blocked ebcidic

Steve

[steven Burgess_2]

Merijn (procura)

I get

syntax error in file /tmp/perl-ea00561 at line 1, next 2 tokens "read STDIN"
Execution of /tmp/perl-ea00561 aborted due to compilation errors.

Any advice ?

Steve

[steven Burgess_2]

Here's the first couple of lines in the file

Zq??????????????????????^Qb^A^W^A^G????^N??(^B^A??????????????K??????K????????K??????????K????????K??????K????????(^B??????????????K??????K????????K??????????K????????K??????K????????Z^\??????????????????????
!AZ'????^A????????????????^Qb@??????????????????????????^Fc^AZ2????^C??????????????????????????@{??@@@@@@@@@@@@@@@@^H??^B??^A??Z^U????^D????????????????

I have just spoken to the sender of the file, they have advised that it is a 'print image file'

a goose chase springs to mind

Steve

[melvyn burnard]

Oh dear, that was not nice of them.
Guess it proved an interesting thread though ;-]

[Carlos Fernandez Riera]

Is the file in a tape or have been transnmited by ftp?

There was a previous online tranaslation done when transmited?

1- It is a tape:
dd if=file_vms of=file_ux bs=32760
# no conversion is done.

The output form this command must be
xxx+0 records in
xxx+0 records out

2- It is ftp'd:

strings file_ux

if you can read any fragment, the file should be previously translated.

And no translation is needed.

Attach the first KB, it you can do it.

[H.Merijn Brand (procura]

What you posted proves it's not the 'Ian' format, so my script wouldn't work anyway.

What version of perl do you use? [ perl4 from /usr/contrib/bin is not good enough ;) ]

what does 'file file.vms' tell you?

is it packed? compressed? encoded?

give us a hex dump of the first 64 bytes.

Enjoy, have FUN! H.Merijn

[Ian Box]

Running a dd conv=ascii on the sample data gives:

!M-;LyFM-^?M-^?M-^?M-^?M-^?M-^?M-^?M-^?;M-)M-2;M- ;M-/;M- ;M-&7M-Q;+M-`M-^H;M-!;
M- CPW2520.INH.I001.F0001.ACIF.RES.TEMPM-^H;M-!CPW2520.INH.I001.F0001.ACIF.OUT.T
EMP!;*LyM-lT1V10500M-^NM-^AM- !^WM-^CM-^SM-^YM-^[Ly;M- T1V10500;M-)M-2 943
5000000000;M-%M-3;M- !^VLw;M-"INTERNATIONAL #5 ;M-'M-g;M-!M-a;
M- 4!;M--Lx;M-#SP010000M-^N

Which has some readable text in it, so that confirms that the source was EBCDIC, but if the data is a print data stream then it will consist of both text and binary data, so any conversion to ASCII is doomed to failure.

Ian