Formatting of text EDI acknowledgement file..

[Kenneth Banyas_1]

I figure I should be able to do this, just don't know which tool.
I have a text EDI ack file. One long stream of text.
However, certain sequences of letters signify, for example, beginning of a record (ST), BAK, addressee (N1), line item (P01), transaction totals (CTT) and so on.
I have been formatting these files in Windows Notepad by inserting carriage returns at these sequences.
This produces readable output for the business analyst.
Surely this can be automated?
I would like to run a script to format the file, then email it.
I am fairly proficient in shell scripting, but I don't think Bourne/POSIX will do it for the formatting.
Neither will awk.
I am thinking sed, which I have never used, or Perl.
Anyone done anything like this?
Point me in the right direction?
Raw EDI text files can be 1K to 50K in size.
Sample 1K file is attached.
Thanks,
Ken Banyas

[curt larson_1]

the only "common" tool that i know that can handle a line length of 50k is perl.

anything else your going to have to break it up into chunks, process the chunks and having to deal with chunk boundaries, then pasting back together. uggh

[Rodney Hills]

perl would be a good tool, and if you set variable $/ to the character(s) that represent a record seperator, you could do record processing

HTH

-- Rod Hills

[A. Clay Stephenson]

Perl is a good choice but another tool is the Gnu version of awk. Unlike standard awk, gawk dynamically allocates memory for input lines and thus the input records can be arbitrarily large.

http://hpux.connect.org.uk/hppd/hpux/Gnu/gawk-3.1.3/

As a Plan C: Develop in gawk (it's generally a much shallower learning curve that that of Perl) and then run a2p to convert your awk code to Perl.

[Hein van den Heuvel]

I would probably go perl and 'binmode' on this problem. Just tell perl to read 'chunks' and look for patterns in the chunks. Try this perl script for size:

$file = shift or die "Please provide filename";
open (FILE, "<$file") or die "Could not open $file";
binmode (FILE);
while ($len = read FILE, $chunk, 1024) { # any size goes
$_ .= $chunk; # glue to remainder
while (/ST~|N1~|PO1~|CTT~/) {
print "$`\n$&"; # print pre-match, newline and match
$_ = $'; # continue work on post-match
}
print "$_\n"; # last bit
}


- I think I got the boundaries right (Seperator accross two chuncks being read)
- You can run this on hpux or the pc.

hth,
Hein.

[Kenneth Banyas_1]

Thank you all for your input.
Solution turned out to be very simple.
No awk, no Perl (although I spent many hours playing with Perl scripting)

Shell script:
for file in *.edi
do
if [ -f "$file" ]
then
tr "*" "\012" < "$file" > "$file".txt
echo "$file".txt
fi
done

tr command. The * is the character in the EDI raw text file which ends a data segment. The \012 inserts the newline.