topic Re: shell script in Operating System - HP-UX

shell script

vas bolpali — Tue, 07 Jan 2003 20:14:25 GMT

we have a data file as
followes. and bad records starts with #@#

$cat data.txt
ABCD 2044440A0023C637
ABCD 2044440B0023C638
#@# 23C63A - 3rd record is bad
ABCD 63744440X023C63B
ABCD 22044440Y023C63C
ABCD 2044740B0023C638
#@# 23C63A - 7th record is bad
ABCD 2033440A0023C637

->| |<-

i need to write a script to log
the info in a file (error.log)

1. which will tell/display the position of
bad record (starts with #@#)

2. and i need to know the
before after records of bad record.

3. I need only from 8 to 12 (length) of those
before and after records ( as shown in arrows)

basically, I have to generate the error log
for the above data file as followes.

$cat error.log
postion before after
3 2044 6374
7 2044 2033

check the attachment for datafile .
Thnx in advance
Vasu

Re: shell script

harry d brown jr — Tue, 07 Jan 2003 20:52:08 GMT

how about something like this:

#!/usr/bin/ksh
awk 'BEGIN {prevhit=0;prevline="";lineno=0}
{
lineno+=1;
if (match($0,/^\#\@\#/)) {
print lineno-1,prevline;
print lineno,$0;
prevhit=1;
} else {
if (prevhit) {
print lineno,$0;
prevhit=0;
}
}
prevline=$0;
}
END {if (prevhit) {print "endofdatafile";}}'

live free or die
harry

Re: shell script

harry d brown jr — Tue, 07 Jan 2003 21:09:01 GMT

Damn this site is very slow today! Anyways I forgot to add the out sample:

# cat data.txt | ./data.awk
2 ABCD 2044440B0023C638
3 #@# 23C63A - 3rd record is bad
4 ABCD 63744440X023C63B
6 ABCD 2044740B0023C638
7 #@# 23C63A - 7th record is bad
8 ABCD 2033440A0023C637

live free or die
harry

Re: shell script

vas bolpali — Tue, 07 Jan 2003 22:07:41 GMT

Thnx for the reply.
you are almost close

DATA=data.txt
ERR=error.log

the error.log must have 3 columns.

1 column- bad record position in the data file
2 column- before record ( may be full or substr)
3 column- after record ( may be full or substr)

the error.log out put should be like this

3 2044 6374
7 2044 2033

vasu.
see the exact datafile in the attachment

Re: shell script

vas bolpali — Tue, 07 Jan 2003 22:08:15 GMT

Re: shell script

Sridhar Bhaskarla — Tue, 07 Jan 2003 23:57:10 GMT

Hi Vasu,

Try this one. Replace datafile with your file and logfile with the output file. Also change cut statement based on how many chars you want.

-Sri

#!/usr/bin/ksh
DATA=datafile
LOG=logfile
LINE=1
printf "%-10.10s %-10.10s %-10.10s \n" POSITION BEFORE AFTER > $LOG
printf "%-10.10s %-10.10s %-10.10s \n" -------- ------ ----- >> $LOG
printf "\n" >> $LOG
while read ENTRY
do
PREV=0
NEXT=0
echo $ENTRY |grep "#@#"
if [ $? = 0 ]
then
(( PREV = $LINE - 1 ))
(( NEXT = $LINE + 1 ))
POSITION="$LINE"
BEFORE=$(sed -n ''$PREV'p' $DATA |cut -c 6-9)
AFTER=$(sed -n ''$NEXT'p' $DATA|cut -c 6-9)
printf "%-10d %-10d %-10d \n" $LINE $BEFORE $AFTER >> $LOG
fi
(( LINE = $LINE + 1 ))
done < $DATA

Re: shell script

Rodney Hills — Wed, 08 Jan 2003 00:01:23 GMT

Create a perl script-

$good=" *BOF";
while(<>) {
chomp;
if (/^#\@#/) { push(@bad,$.); }
dump();
$curr=substr($_,7,5);
$good=$curr;
}
$curr=" *EOF";
dump();

sub dump {
if (scalar @bad) {
foreach $badline (@bad) {
print "$badline $good $curr\n";
}
undef @bad;
}
return;
}

----------------
Then run-
perl yourscript datafile >error.log

This will handle if their are mutiple bad lines in a row.
It will also handle if the first or last line is bad.

HTH

-- Rod Hills

Re: shell script

Sridhar Bhaskarla — Wed, 08 Jan 2003 00:01:28 GMT

couple of minor changes.

Replace

echo $ENTRY |grep "#@#"

with

echo $ENTRY |grep "#@#" > /dev/null 2>&1

Remove

POSITION="$LINE"

I didn't use it.

-Sri

Re: shell script

Rodney Hills — Wed, 08 Jan 2003 00:17:55 GMT

Ignore previous perl script, this is the correct script I meant to enter-

$good=" *BOF";
while(<>) {
chomp;
if (/^#\@#/) { push(@bad,$.); }
else {
$curr=substr($_,7,5);
dump();
$good=$curr;
}
}
$curr=" *EOF";
dump();

sub dump {
if (scalar @bad) {
foreach $badline (@bad) {
print "$badline $good $curr\n";
}
undef @bad;
}
return;
}

-- Rod Hills

Re: shell script

john korterman — Wed, 08 Jan 2003 12:05:23 GMT

Hi Vasu,
try the attached script, using your input file as $1.
You can modify the STRING variable if the requested chars are not correct.

regards,
John K.

Re: shell script

Jean-Louis Phelix — Wed, 08 Jan 2003 12:55:13 GMT

hi,

Perhaps a shorter one ...

#!/usr/bin/sh
nl -s^V data.txt | awk -F^V '{
if (match($2,/#@#/))
{
printf "%d %s", $1, substr(prev, 8, 5)
getline
printf " %s\n", substr($2, 8, 5)
}
prev=$2
}' > error.log

Regards.

Re: shell script

harry d brown jr — Wed, 08 Jan 2003 13:32:11 GMT

Vasu,

Try this, I modified the data a little in case the FIRST or LAST record was bad:

# cat data.txt
#@# 23C63A - 1 st record is bad
ABCD 2044440B0023C638
#@# 23C63A - 3rd record is bad
ABCD 63744440X023C63B
ABCD 22044440Y023C63C
ABCD 2044740B0023C638
#@# 23C63A - 7th record is bad
ABCD 2033440A0023C637
ABCD 2044740B0023C638
#@# 23C63A - 10th record is bad

# cat data.awk
#!/usr/bin/ksh
awk 'BEGIN {
prevline=""; prevdata=""; nextdata=""; lineno=0; badlineno=0;
}
{
lineno+=1;
if (match($0,/^\#\@\#/)) {
badlineno=lineno;
prevdata=substr(prevline,6,4);
if (prevdata == "") prevdata="none";
} else {
if (badlineno) {
nextdata=substr($0,6,4);
print badlineno,prevdata,nextdata;
badlineno=0;
prevdata="";
nextdata="";
}
}
prevline=$0;
}
END {if (badlineno) {print badlineno,prevdata,"none";}}'

# cat data.txt | ./data.awk
1 none 2044
3 2044 6374
7 2044 2033
10 2044 none
#

live free or die
harry

Re: shell script

vas bolpali — Wed, 08 Jan 2003 18:56:04 GMT

I gave only the basic info .

but john korterman 's script does exactly what i need in full .

Sridhar gave me exactly , what i asked ,
but in reality it went up to 80% only .

and thanks all of you .

Vasu.

Re: shell script

vas bolpali — Tue, 14 Jan 2003 20:49:40 GMT

hi John,
The shell script is not handling , if all the records are bad.
check the attachment for tha data file.

Re: shell script

harry d brown jr — Tue, 14 Jan 2003 21:05:01 GMT

Try this:

#!/usr/bin/ksh
awk 'BEGIN {
prevline=""; prevdata=""; nextdata=""; lineno=0; badlineno=0; goodlineno=0;
}
{
lineno+=1;
if (match($0,/^\#\@\#/)) {
badlineno=lineno;
prevdata=substr(prevline,6,4);
if (prevdata == "") prevdata="none";
} else {
goodlineno=lineno;
if (badlineno) {
nextdata=substr($0,6,4);
print badlineno,prevdata,nextdata;
badlineno=0;
prevdata="";
nextdata="";
}
}
prevline=$0;
}
END {
if (badlineno) {
if (goodlineno) {
print badlineno,prevdata,"none";
} else {
print "All lines are bad";
}
}
}'

live free or die
harry

Re: shell script

vas bolpali — Tue, 14 Jan 2003 21:16:26 GMT

no ,
if all records are bad , still i need err
report/output like this .
(assuming total 50 ,
and all are bad)

1 null null
2 null null
3 null null
......
50 null null

Re: shell script

Rodney Hills — Tue, 14 Jan 2003 21:28:27 GMT

If you go back to my perl script above, it will handle when all lines are bad.

1 *BOF *EOF
2 *BOF *EOF
...
50 *BOF *EOF

The only difference is I used *BOF and *EOF to represent beginning file and ending file.

-- Rod Hills

Re: shell script

vas bolpali — Tue, 14 Jan 2003 22:44:36 GMT

hi Rod
u r perl script (perlx) is giving errors

$ perl perlx data.txt > error.log

syntax error in file perlx at line 8, next 2 tokens "dump("

syntax error in file perlx at line 13, next 2 tokens "dump("

Execution of perlx aborted due to compilation errors.

Vasu

Re: shell script

Rodney Hills — Tue, 14 Jan 2003 23:10:34 GMT

Whoops, I think "dump" is a reserved name in perl. Try renaming "dump" to "dispit".

-- Rod Hills

Re: shell script

harry d brown jr — Wed, 15 Jan 2003 13:43:02 GMT

Try this attached awk script:

live free or die
harry