sed - deletion of multiple lines in a file

[Eric Lipede]

Hi

Scenario
========
I have a file with the word root appearing randomly in it. Ie on line 7 , 13 , 27 etc.
Id like to delete the line containing "root" and 2 lines above it and 2 lines below it (in a script). Any ideas?
thks

[James R. Ferguson · Accepted solution]

Hi Eric:

# cat snip2
#!/usr/bin/perl
use strict;
use warnings;
my $skip = 0;
my @lines;
while (<>) {
chomp;
if (/root/) {
$skip=2;
pop @lines for (1..$skip);
next;
}
if ($skip-- <= 0) {
push(@lines,$_);
}
}
print "$_\n" for (@lines);
1;

...run as:

# ./snip2 file

Regards!

...JRF...

[Hein van den Heuvel]

Here is a silly one-liner.

perl -ne 'print $l3 if --$root<0; $l3=$l2; $l2=$l1;$l1=$_; $root=5 if /root/}{print $l3 if --$root<0; print $l2 if --$root<0; print $l1 if --$root<0' tmp.txt

In slow motion...

while (<>) {
print $l3 if --$root<0; # only if no root in near future
$l3=$l2; # delay line...
$l2=$l1;
$l1=$_;
$root=5 if /root/
}
# deal with last lines;
print $l3 if --$root<0;
print $l2 if --$root<0;
print $l1 if --$root<0;

So many ways to do it.

This probaly codes more tightly using a simple 3 line array indexed by ( $. % 3 )

Cheers,
Hein.

[Eric Lipede]

Hi Guys
points a-coming ..great perl solutions ....only we arent using perl could i bother you for the awk / sed equivalents - many thks


****** grep for the stars

[Hein van den Heuvel]

perl -ne 'print $l[($.-3)%3] if --$root<0; $l[$.%3]=$_; $root=5 if /root/} {for $i ($.-2 .. $.) { print $l[$i%3] if --$root<0}' tmp.txt

Same method of course also works in awk, but you have to not try to print for the first 3 lines.

$ awk '(--r<0 && NR>3) {print l[(NR-3)%3]} /root/ {r=5} {l[NR%3]=$0} END {for (i=NR-2; i<=NR; i++) {if (--r<0) {print l[i%3]}}}'


perl array solution pretty printed....

while (<>) {
print $l[($.-3)%3] if --$root<0;
$l[$.%3]=$_;
$root=5 if /root/;
}
for $i ($.-2 .. $.) {
print $l[$i%3] if --$root<0
}

Hein

[James R. Ferguson]

Hi (again):

TMTOWTDI.

...JRF...

[Eric Lipede]

Hi Hein
Im trying to modify the awk script to:
delete the previous 9 lines and 9 lines after the found text "root is not locked"
(including the found text)

file is....
*************************************************
Cron: The previous message is the standard output
and standard error of one of your crontab commands:

/hpuxsw/utilities/health/unlr.sh


From SYSTEM-ALERT@rx86 Thu Jun 26 15:00:01 BST 2008
Received: (from root@localhost)
by x.sw.corp (8.11.1 (PHNE_35485)/8.11.1) id m5QE01218793
for root; Thu, 26 Jun 2008 15:00:01 +0100 (BST)
Date: Thu, 26 Jun 2008 15:00:01 +0100 (BST)
From: SYSTEM-ALERT@rx86
Message-Id: <200806261400.m5QE01218793@lonunix2.sw.corp>
Subject: cron

root is not locked


*************************************************
Cron: The previous message is the standard output
and standard error of one of your crontab commands:

/hpuxsw/utilities/health/unlr.sh





...not having much joy ! :-(

[James R. Ferguson]

Hi Eric:

Well, you could modify the Perl script I suggested in response to your last query.

Change:

$skip=2;

to

$skip=9;

...and anchor (with a caret) the match for the word "root" like:

if (/^root/) {

As I said, TMTOWTDI.

Regards!

...JRF...

[James R. Ferguson]

Hi (again) Eric:

I hope that your lack of credit for my efforts isn't a case of "Condemnant quod non intellegunt" but rather a simple, unintentional oversight.

...JRF...

[Hein van den Heuvel]

Eric>> only we arent using perl could i bother you for the awk / sed equivalents

I had need seen this when i volunteerd the awk solution. I probably would not have done the awk alternate had I read that :^).

Please consider ANY solution when kindly offered, and reward the effort.
Admittedly you did mention 'sed' in the subject.

Let's face is you ( your peers ) could not solve it in the language of your choice, so clearly (for me) that language holds no (maintenance) advantage.
Any non-sed solution is surely just as hard (easy) to understand / maintain whether it be perl or awk or shell or any other 'likely to be available' language.

Anyway...

Eric>> Im trying to modify the awk script to:

Well, my first awk and perl suggestions would be too tedious to grow the 'delay line / bucket brigade'

The second awk suggestion, with the array, is more flexible, but was spinkled with delete window dependend contants.

James's solution is much cleaner in that he need to set the number of lines to skip just once: $skip=2;
So just one edit needed to change to 9 lines.

His solution build a potentiall very large array of lines to keep, so he can print it all at the end. Note, his solution might miss a re-occurance of 'root' in the deleted lines which may or might not be desirable. With the newer specification it is clear that is a moot point thought.

My solution has a one-on-one print but 'lagging' by N lines. So it has the drwaback of needing to fix what is still in the queue on reading the last record.

Anyway here is an (awk) version of the script with the 'Window' exposed as variable:

------------- filter.awk -----------
BEGIN { Window = 9 + 1}
(--Count<0 && NR > Window ) {print line[(NR - Window)%Window]}
/root is not locked/ {Count=2*Window - 1}
{line[NR%Window]=$0}
END {
for (i=NR-Window; i i++;
if (--Count<0) {
print line[i%Window];
}
}
}
------------------

awk -f filter.awk

cheers,
Hein.

[Dennis Handly]

>I'd like to delete the line containing "root" and 2 lines above it and 2 lines below it (in a script).

The gnu grep has a -C option to list the N lines before and after a match. I don't know if the -v option will remove them?

[James R. Ferguson]

HI Eric:

You said, "only we arent using perl" and yet Perl was wholly appropriate in an earlier thread of yours:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1199213

Credit for the help I offered you therein would be appreciated, too. Thanks.

Regards!

...JRF...

[Eric Lipede]

I of course thank all of you for your invaluable help. I cant help however be slighty bemused over the preciousness of the use of languages. Surely, the requestor (me) has the choice of language and more to the point, regardless of the "appropriateness" of a solution, that solution may not be the right one or indeed the best. For those of you that have worked in big organizations, the choice of one language over another sometimes may range from political to out right ignorance. Whatever the case, just because Perl is appropriate does not make it the best to use. I love Perl for the detractors out there btw. Its just that in THIS case , awk and sed would be the route to go down ...as i indicated in my original request - further adding credance to the "theres method in the maddness". I asked for a sed solution and i got it - thks again. ....i do love the "and another thing" ....these threads tend to produce.

Thks again - and points awarded!

[Eric Lipede]

...btw James - I did use Perl at the Company I worked at when that thread was generated. The Company I work at now doesnt.

...its ALL good.

;-)

[James R. Ferguson]

Hi (again) Eric:

Your closing points are well taken. I'm not (usuallly) one to fall into a Holy War over languages, despite my love for Perl.

Your comments shed light on the "madness" indeed, and I appreciate you sharing them!

NO points for this, please. As you said, "it's ALL good"!

Regards!

...JRF...

[Dennis Handly]

>ME: I don't know if the -v option will remove them?

It appears that -v and -C## fails to play nicely together. :-(

[Eric Lipede]

Hi
this is working quite sweetly now ...

ROOTMAIL="/var/mail/root"
SCRATCH="/hpuxsw/utilities/health/logs/manage_mail.`date +%d%m%y-%T`"
let i=0
export STATE=CONTINUE

clean_file_1()
{
while [[ "$STATE" = "CONTINUE" ]]
do
cat $ROOTMAIL | while read
do
let i=$i+1
if
echo "${REPLY}" | grep "root is not locked" > /dev/null 2>&1
then
if [ "$i-9" -lt 1 ]
then
echo "Value of FROM = $FROM"
echo "Error"
exit
else
let FROM=$i-9;export FROM
let TO=$i+9;export TO
fi
sed "$FROM,$TO"d $ROOTMAIL > $SCRATCH
cat $SCRATCH > $ROOTMAIL
fi
done
check_file_1
done
}

check_file_1()
{
if
grep "root is not locked" $ROOTMAIL > /dev/null 2>&1
then
export STATE=CONTINUE
let i=0
else
export STATE=STOP
if [ -f $SCRATCH ]
then
rm $SCRATCH
fi
fi
}


#############
# Start
#############

clean_file_1

[mobidyc]

Hello,

you can use GNU grep like this:
/usr/local/bin/grep -A 2 -B 2 root file

#> cat /tmp/file
1
2
3
root
4
5
6
7
8

#> /usr/local/bin/grep -A 2 -B 2 root /tmp/file
2
3
root
4
5

#>

--
Regards,
Cedrick Gaillard

[mobidyc]

Hello,

don't forget to add the -v flag for delete the lines you don't want:

/usr/local/bin/grep -v -A 2 -B 2 root /tmp/file > /tmp/file.tmp && mv /tmp/file.tmp /tmp/file

--
Regards,
Cedrick Gaillard

[Dennis Handly]

>mobidyc: don't forget to add the -v flag

My 2.5.1 doesn't handle -v with either -C or -A/-B.

[James R. Ferguson]

Hi (again) Eric:

If I might suggest a few minor tweaks to your script:

1. Eliminate the unnecessary 'cat' process to read your input. Instead of:

cat $ROOTMAIL | while read

...change to:

while read
do
...
done < ${ROOTMAIL}

2. Don't read a file only to replace another by it. Instead of:

cat $SCRATCH > $ROOTMAIL

...simply do:

mv ${SCRATCH} ${ROOTMAIL}

3. Since it appears that this is one script you don't need to 'export' variables like 'STATE".

Regards!

...JRF...

[Eric Lipede]

tweaks are always good - thks and points awarded !

[Eric Lipede]

Problem resolved