Perl Help

[Robert True]

I am trying to manipulate a data file to retrieve information for down-stream processing.

The records are 1563 bytes long, start with an ascii 'soh' (001) and end with a 'C'. The record id is the first data field and is 21 bytes long.

I want to ignore records with id = all '9's., but I do need to include the 'soh' as the first byte for the downstream app. I also need to add an newline after the end of record 'C'.

The following snippet works, except for trying to match the $id with the 'soh' included as the first charactor in the buffer, IE: if I slew the reads so that the 'soh' is the last char of the previous record, it will drop the ids = all 9s.

The basic problem is how do I get the "if $id != 'soh'(9x21)" statement to catch only the good records? or what sprintf format do I need to tack the 'soh' back on in the right place. I have tried several formats with no success.

Hope that states what I am trying to do - skills with Perl are a bit ruff, in case you hadn't guessed.



Snippet:

while (read(I_FILE, $buf, 1563)) {
$id = sprintf("%-21s",$buf);
if ($debug) {
printf("%s\n",$id );
if ($id == "999999999999999999999") {
printf("%s",$buf);
print("\n");
}
}
if (!$debug) {
if ($id != "999999999999999999999") {
printf(O_FILE "%s",$buf);
print(O_FILE "\n");
}
}
}


Thanks,
Rt.

[H.Merijn Brand (procura · Accepted solution]

Fixed records are easiest to deal with using pack and unpack.
sprintf is a not so good choice here. Either substr () or unpack is better. Most severe error you made is a numeric comparison between ID and "9999..." which both extend the maximum storable integer value, even in 64bit environment. You will have to deal with the keys/ID's in string context, using 'eq' instead of '=='

If every record is 1563 bytes, with 21 bytes to introduce the ID,

while (read (I_FILE, $buf, 1563)) {
($id) = unpack "A21", $buf;
if ($id eq "9"x21){
# yadayadayada
next;
}
# this for all in "99999...." stuff
print O_FILE "$buf\n";
}

Enjoy, Have FUN! H.Merijn

[Hein van den Heuvel]

Procura is right the pack/unpack for fixed length of course. But don't rule out the simble 'substr': $id = substr($buf,1,21);

Here are a few more ideas for a solution

- switch to binmode ?!
- use a regexp to find proper record
. starts "^" with soh "\x01" for hex 1
. remember "(.{21})" 21 characters in $1
. haveing a "C" just before the end "$"


open (I_FILE, "binmode (I_FILE);
while (read (I_FILE, $_, 1536)) {
$i++;
if (/^\x01(.{21}).*C$/) {
$id = $1;
if ($id eq "9"x21) {
print "record $i : skipped\n";
} else {
print (O_FILE, "$_\n");
}
} else {
print "record $i : bad head or tail\n";
}
}



if you are garantueed the contents is good, you could also just test for the soh followed by 9999 and skip those prinnting everything else using something like:

while (read (I_FILE, $_, 1536)) {
next if (/^\x{01}9{21}/)
print (O_FILE, "$_\n");
}


or even replace the exact match for soh with the \x{01} by just any-char: (/^.9{21}/)


Have fun,
Hein.

[Robert True]

Thanks guys! You're right about the unpack, it was the key to completing the record picking portion, and I now understand both unpack and substr a bit, after reading the man page along with your examples, which give me more ideas for this and some other projects I am currently working on. Trying to ramp up on Perl skills.

The '==' instead of 'eq' is a bad habit from sloppy 'ksh' work that I need to rid myself of. Most of the time in ksh it will work, but it is a bad habit.

Thanks again.

Rt.