topic Re: file conversion using dd in Operating System - HP-UX

file conversion using dd

steven Burgess_2 — Thu, 10 Apr 2003 07:01:15 GMT

Hi all

I have a file that has the following block size and record length

REMOTE_SYSTEM_TYPE=MVS
RECORD_LENGTH=32756
BLOCK_SIZE=32760

The file has come from remote VMS and looks as though it has been blocked for space purposes.

It comes through as awk program text as the HP doesn't know what type of file it is

PROD.tsysuk-edouk-stmts.20030307231003: awk program text

I am trying to convert the file to ascii using dd with the following permitations

dd if=PROD.tsysuk-edouk-stmts.20030307231003 conv=ascii bs=32760 of=ste.new

dd if=PROD.tsysuk-edouk-stmts.20030307231003 conv=ascii cbs=32760 of=ste.new

dd if=PROD.tsysuk-edouk-stmts.20030307231003 conv=ascii bs=110 of=ste.new

dd if=PROD.tsysuk-edouk-stmts.20030307231003 conv=ascii cbs=110 of=ste.new

Plus others. The cbs=110 is giving me some readable text but not enough.

Can anyone advise further ?

Thanks in advance

Steve

Re: file conversion using dd

melvyn burnard — Thu, 10 Apr 2003 07:09:11 GMT

Steven
try your first attempt again, but use ibs=32760 as opposed to bs=32760

More often than not, this is a long drawn out play with all hte variables until you get what you want out.

Re: file conversion using dd

steven Burgess_2 — Thu, 10 Apr 2003 07:55:50 GMT

Hi

Struggling -

Steve

Re: file conversion using dd

melvyn burnard — Thu, 10 Apr 2003 08:16:19 GMT

ok, so now try:
dd if=PROD.tsysuk-edouk-stmts.20030307231003 conv=ascii ibs=32760 cbs=32756 of=ste.new

Re: file conversion using dd

steven Burgess_2 — Thu, 10 Apr 2003 08:30:55 GMT

Hi Melvyn

Some ascii char but the rest is just control characters

Thanks

Steve

Re: file conversion using dd

melvyn burnard — Thu, 10 Apr 2003 08:54:15 GMT

Ouch, this may be a painful process. Hopefully there may be others in the forum who might be able to shed more light on this for you.

Re: file conversion using dd

Jean-Louis Phelix — Thu, 10 Apr 2003 09:36:39 GMT

Hi Steve,

A long time ago I had to play this kind of game ... But I don't known VMS. Are you sure source coding is EBCDIC ? Sometimes I also had to play with conv=swab (to swap bytes) or conv=unblock to convert from fixed-length to variable-length.

Regards.

Re: file conversion using dd

H.Merijn Brand (procura — Thu, 10 Apr 2003 10:01:39 GMT

MVS != VMS :)

What is it you want to achieve? Read blocks of 32760 bytes, strip trailing spaces and write to (ASCII) file with newlines instead?

can you make the first 100k of the file available as URL, so we can have a look? I bet it's doable with perl, whatever you want to achieve :]

Enjoy, have FUN! H.Merijn

Re: file conversion using dd

Ian Box — Thu, 10 Apr 2003 10:18:05 GMT

I'm guessing here, but if the blocksize is 32760 and the record length is 32756 then you've probably got what the IBM world calls variable blocked records.

If this is the case, then the first 2 bytes of each block will hold the binary length of the block, and then each record within the block will have a 2 byte binary record length at the beginning. In each case the third and fourth bytes should be zero (they are used for something, but it's so long since I did anything with these records that I can't remember what). Each record can be up to 32756 bytes, but could be much shorter.

Sorry, I don't know if there is any standard utility on HP-UX which will re-block them.

How did you get the data? Was it ftp'd from an MVS mainframe? If so there should be some options on the ftp transfer that might allow reblocking. A 'help site' in ftp might provide assistance.

HTH

Ian

Re: file conversion using dd

H.Merijn Brand (procura — Thu, 10 Apr 2003 10:50:50 GMT

If Ian is right, try this:

#perl -le'while(read STDIN,$_,32760){print unpack"s/a",$_}' file.vms >file.txt

variations of s/a to n/a or v/a (big- vs. little-endian numbers) could help

Enjoy, have FUN! H.Merijn

Re: file conversion using dd

steven Burgess_2 — Thu, 10 Apr 2003 11:47:43 GMT

Hi everyone

Thanks for all your input. I have had a really busy morning , will try out your suggestions this afternoon and get back to you

Thanks again

Steve

Re: file conversion using dd

steven Burgess_2 — Thu, 10 Apr 2003 13:49:59 GMT

Hi

And yes - it's from mvs , not vms (doh!) and variable blocked ebcidic

Steve

Re: file conversion using dd

steven Burgess_2 — Thu, 10 Apr 2003 13:56:12 GMT

Merijn (procura)

I get

syntax error in file /tmp/perl-ea00561 at line 1, next 2 tokens "read STDIN"
Execution of /tmp/perl-ea00561 aborted due to compilation errors.

Any advice ?

Steve

Re: file conversion using dd

steven Burgess_2 — Thu, 10 Apr 2003 14:00:34 GMT

Here's the first couple of lines in the file

Zq??????????????????????^Qb^A^W^A^G????^N??(^B^A??????????????K??????K????????K??????????K????????K??????K????????(^B??????????????K??????K????????K??????????K????????K??????K????????Z^\??????????????????????
!AZ'????^A????????????????^Qb@??????????????????????????^Fc^AZ2????^C??????????????????????????@{??@@@@@@@@@@@@@@@@^H??^B??^A??Z^U????^D????????????????

I have just spoken to the sender of the file, they have advised that it is a 'print image file'

a goose chase springs to mind

Steve

Re: file conversion using dd

melvyn burnard — Thu, 10 Apr 2003 14:04:59 GMT

Oh dear, that was not nice of them.
Guess it proved an interesting thread though ;-]

Re: file conversion using dd

Carlos Fernandez Riera — Thu, 10 Apr 2003 14:19:33 GMT

Is the file in a tape or have been transnmited by ftp?

There was a previous online tranaslation done when transmited?

1- It is a tape:
dd if=file_vms of=file_ux bs=32760
# no conversion is done.

The output form this command must be
xxx+0 records in
xxx+0 records out

2- It is ftp'd:

strings file_ux

if you can read any fragment, the file should be previously translated.

And no translation is needed.

Attach the first KB, it you can do it.

Re: file conversion using dd

H.Merijn Brand (procura — Fri, 11 Apr 2003 07:20:07 GMT

What you posted proves it's not the 'Ian' format, so my script wouldn't work anyway.

What version of perl do you use? [ perl4 from /usr/contrib/bin is not good enough ;) ]

what does 'file file.vms' tell you?

is it packed? compressed? encoded?

give us a hex dump of the first 64 bytes.

Enjoy, have FUN! H.Merijn

Re: file conversion using dd

Ian Box — Fri, 11 Apr 2003 08:49:57 GMT

Running a dd conv=ascii on the sample data gives:

!M-;LyFM-^?M-^?M-^?M-^?M-^?M-^?M-^?M-^?;M-)M-2;M- ;M-/;M- ;M-&7M-Q;+M-`M-^H;M-!;
M- CPW2520.INH.I001.F0001.ACIF.RES.TEMPM-^H;M-!CPW2520.INH.I001.F0001.ACIF.OUT.T
EMP!;*LyM-lT1V10500M-^NM-^AM- !^WM-^CM-^SM-^YM-^[Ly;M- T1V10500;M-)M-2 943
5000000000;M-%M-3;M- !^VLw;M-"INTERNATIONAL #5 ;M-'M-g;M-!M-a;
M- 4!;M--Lx;M-#SP010000M-^N

Which has some readable text in it, so that confirms that the source was EBCDIC, but if the data is a print data stream then it will consist of both text and binary data, so any conversion to ASCII is doomed to failure.

Ian