topic Re: sed - deletion of multiple lines in a file in Operating System - HP-UX

sed - deletion of multiple lines in a file

Eric Lipede — Tue, 01 Jul 2008 14:41:29 GMT

Hi

Scenario
========
I have a file with the word root appearing randomly in it. Ie on line 7 , 13 , 27 etc.
Id like to delete the line containing "root" and 2 lines above it and 2 lines below it (in a script). Any ideas?
thks

Re: sed - deletion of multiple lines in a file

James R. Ferguson — Tue, 01 Jul 2008 15:21:41 GMT

Hi Eric:

# cat snip2
#!/usr/bin/perl
use strict;
use warnings;
my $skip = 0;
my @lines;
while (<>) {
chomp;
if (/root/) {
$skip=2;
pop @lines for (1..$skip);
next;
}
if ($skip-- <= 0) {
push(@lines,$_);
}
}
print "$_\n" for (@lines);
1;

...run as:

# ./snip2 file

Regards!

...JRF...

Re: sed - deletion of multiple lines in a file

Hein van den Heuvel — Tue, 01 Jul 2008 15:59:58 GMT

Here is a silly one-liner.

perl -ne 'print $l3 if --$root<0; $l3=$l2; $l2=$l1;$l1=$_; $root=5 if /root/}{print $l3 if --$root<0; print $l2 if --$root<0; print $l1 if --$root<0' tmp.txt

In slow motion...

while (<>) {
print $l3 if --$root<0; # only if no root in near future
$l3=$l2; # delay line...
$l2=$l1;
$l1=$_;
$root=5 if /root/
}
# deal with last lines;
print $l3 if --$root<0;
print $l2 if --$root<0;
print $l1 if --$root<0;

So many ways to do it.

This probaly codes more tightly using a simple 3 line array indexed by ( $. % 3 )

Cheers,
Hein.

Re: sed - deletion of multiple lines in a file

Eric Lipede — Tue, 01 Jul 2008 17:24:27 GMT

Hi Guys
points a-coming ..great perl solutions ....only we arent using perl could i bother you for the awk / sed equivalents - many thks

****** grep for the stars

Re: sed - deletion of multiple lines in a file

Hein van den Heuvel — Tue, 01 Jul 2008 17:50:35 GMT

perl -ne 'print $l[($.-3)%3] if --$root<0; $l[$.%3]=$_; $root=5 if /root/} {for $i ($.-2 .. $.) { print $l[$i%3] if --$root<0}' tmp.txt

Same method of course also works in awk, but you have to not try to print for the first 3 lines.

$ awk '(--r<0 && NR>3) {print l[(NR-3)%3]} /root/ {r=5} {l[NR%3]=$0} END {for (i=NR-2; i<=NR; i++) {if (--r<0) {print l[i%3]}}}'

perl array solution pretty printed....

while (<>) {
print $l[($.-3)%3] if --$root<0;
$l[$.%3]=$_;
$root=5 if /root/;
}
for $i ($.-2 .. $.) {
print $l[$i%3] if --$root<0
}

Hein

Re: sed - deletion of multiple lines in a file

James R. Ferguson — Tue, 01 Jul 2008 17:57:36 GMT

Hi (again):

TMTOWTDI.

...JRF...

Re: sed - deletion of multiple lines in a file

Eric Lipede — Tue, 01 Jul 2008 18:16:27 GMT

Hi Hein
Im trying to modify the awk script to:
delete the previous 9 lines and 9 lines after the found text "root is not locked"
(including the found text)

file is....
*************************************************
Cron: The previous message is the standard output
and standard error of one of your crontab commands:

/hpuxsw/utilities/health/unlr.sh

From SYSTEM-ALERT@rx86 Thu Jun 26 15:00:01 BST 2008
Received: (from root@localhost)
by x.sw.corp (8.11.1 (PHNE_35485)/8.11.1) id m5QE01218793
for root; Thu, 26 Jun 2008 15:00:01 +0100 (BST)
Date: Thu, 26 Jun 2008 15:00:01 +0100 (BST)
From: SYSTEM-ALERT@rx86
Message-Id: <200806261400.m5QE01218793@lonunix2.sw.corp>
Subject: cron

root is not locked

*************************************************
Cron: The previous message is the standard output
and standard error of one of your crontab commands:

/hpuxsw/utilities/health/unlr.sh

...not having much joy ! :-(

Re: sed - deletion of multiple lines in a file

James R. Ferguson — Tue, 01 Jul 2008 18:28:29 GMT

Hi Eric:

Well, you could modify the Perl script I suggested in response to your last query.

Change:

$skip=2;

to

$skip=9;

...and anchor (with a caret) the match for the word "root" like:

if (/^root/) {

As I said, TMTOWTDI.

Regards!

...JRF...

Re: sed - deletion of multiple lines in a file

James R. Ferguson — Tue, 01 Jul 2008 19:09:49 GMT

Hi (again) Eric:

I hope that your lack of credit for my efforts isn't a case of "Condemnant quod non intellegunt" but rather a simple, unintentional oversight.

...JRF...

Re: sed - deletion of multiple lines in a file

Hein van den Heuvel — Tue, 01 Jul 2008 21:14:32 GMT

Eric>> only we arent using perl could i bother you for the awk / sed equivalents

I had need seen this when i volunteerd the awk solution. I probably would not have done the awk alternate had I read that :^).

Please consider ANY solution when kindly offered, and reward the effort.
Admittedly you did mention 'sed' in the subject.

Let's face is you ( your peers ) could not solve it in the language of your choice, so clearly (for me) that language holds no (maintenance) advantage.
Any non-sed solution is surely just as hard (easy) to understand / maintain whether it be perl or awk or shell or any other 'likely to be available' language.

Anyway...

Eric>> Im trying to modify the awk script to:

Well, my first awk and perl suggestions would be too tedious to grow the 'delay line / bucket brigade'

The second awk suggestion, with the array, is more flexible, but was spinkled with delete window dependend contants.

James's solution is much cleaner in that he need to set the number of lines to skip just once: $skip=2;
So just one edit needed to change to 9 lines.

His solution build a potentiall very large array of lines to keep, so he can print it all at the end. Note, his solution might miss a re-occurance of 'root' in the deleted lines which may or might not be desirable. With the newer specification it is clear that is a moot point thought.

My solution has a one-on-one print but 'lagging' by N lines. So it has the drwaback of needing to fix what is still in the queue on reading the last record.

Anyway here is an (awk) version of the script with the 'Window' exposed as variable:

------------- filter.awk -----------
BEGIN { Window = 9 + 1}
(--Count<0 && NR > Window ) {print line[(NR - Window)%Window]}
/root is not locked/ {Count=2*Window - 1}
{line[NR%Window]=$0}
END {
for (i=NR-Window; i i++;
if (--Count<0) {
print line[i%Window];
}
}
}
------------------

awk -f filter.awk

cheers,
Hein.

Re: sed - deletion of multiple lines in a file

Dennis Handly — Tue, 01 Jul 2008 21:34:26 GMT

>I'd like to delete the line containing "root" and 2 lines above it and 2 lines below it (in a script).

The gnu grep has a -C option to list the N lines before and after a match. I don't know if the -v option will remove them?

Re: sed - deletion of multiple lines in a file

James R. Ferguson — Tue, 01 Jul 2008 22:56:52 GMT

HI Eric:

You said, "only we arent using perl" and yet Perl was wholly appropriate in an earlier thread of yours:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1199213

Credit for the help I offered you therein would be appreciated, too. Thanks.

Regards!

...JRF...

Re: sed - deletion of multiple lines in a file

Eric Lipede — Wed, 02 Jul 2008 03:51:28 GMT

I of course thank all of you for your invaluable help. I cant help however be slighty bemused over the preciousness of the use of languages. Surely, the requestor (me) has the choice of language and more to the point, regardless of the "appropriateness" of a solution, that solution may not be the right one or indeed the best. For those of you that have worked in big organizations, the choice of one language over another sometimes may range from political to out right ignorance. Whatever the case, just because Perl is appropriate does not make it the best to use. I love Perl for the detractors out there btw. Its just that in THIS case , awk and sed would be the route to go down ...as i indicated in my original request - further adding credance to the "theres method in the maddness". I asked for a sed solution and i got it - thks again. ....i do love the "and another thing" ....these threads tend to produce.

Thks again - and points awarded!

Re: sed - deletion of multiple lines in a file

Eric Lipede — Wed, 02 Jul 2008 03:55:44 GMT

...btw James - I did use Perl at the Company I worked at when that thread was generated. The Company I work at now doesnt.

...its ALL good.

;-)

Re: sed - deletion of multiple lines in a file

James R. Ferguson — Wed, 02 Jul 2008 11:20:58 GMT

Hi (again) Eric:

Your closing points are well taken. I'm not (usuallly) one to fall into a Holy War over languages, despite my love for Perl.

Your comments shed light on the "madness" indeed, and I appreciate you sharing them!

NO points for this, please. As you said, "it's ALL good"!

Regards!

...JRF...

Re: sed - deletion of multiple lines in a file

Dennis Handly — Mon, 07 Jul 2008 23:40:17 GMT

>ME: I don't know if the -v option will remove them?

It appears that -v and -C## fails to play nicely together. :-(

Re: sed - deletion of multiple lines in a file

Eric Lipede — Tue, 08 Jul 2008 04:26:18 GMT

Hi
this is working quite sweetly now ...

ROOTMAIL="/var/mail/root"
SCRATCH="/hpuxsw/utilities/health/logs/manage_mail.`date +%d%m%y-%T`"
let i=0
export STATE=CONTINUE

clean_file_1()
{
while [[ "$STATE" = "CONTINUE" ]]
do
cat $ROOTMAIL | while read
do
let i=$i+1
if
echo "${REPLY}" | grep "root is not locked" > /dev/null 2>&1
then
if [ "$i-9" -lt 1 ]
then
echo "Value of FROM = $FROM"
echo "Error"
exit
else
let FROM=$i-9;export FROM
let TO=$i+9;export TO
fi
sed "$FROM,$TO"d $ROOTMAIL > $SCRATCH
cat $SCRATCH > $ROOTMAIL
fi
done
check_file_1
done
}

check_file_1()
{
if
grep "root is not locked" $ROOTMAIL > /dev/null 2>&1
then
export STATE=CONTINUE
let i=0
else
export STATE=STOP
if [ -f $SCRATCH ]
then
rm $SCRATCH
fi
fi
}

#############
# Start
#############

clean_file_1

Re: sed - deletion of multiple lines in a file

mobidyc — Tue, 08 Jul 2008 06:36:41 GMT

Hello,

you can use GNU grep like this:
/usr/local/bin/grep -A 2 -B 2 root file

#> cat /tmp/file
1
2
3
root
4
5
6
7
8

#> /usr/local/bin/grep -A 2 -B 2 root /tmp/file
2
3
root
4
5

#>

--
Regards,
Cedrick Gaillard

Re: sed - deletion of multiple lines in a file

mobidyc — Tue, 08 Jul 2008 06:56:17 GMT

Hello,

don't forget to add the -v flag for delete the lines you don't want:

/usr/local/bin/grep -v -A 2 -B 2 root /tmp/file > /tmp/file.tmp && mv /tmp/file.tmp /tmp/file

--
Regards,
Cedrick Gaillard

Re: sed - deletion of multiple lines in a file

Dennis Handly — Tue, 08 Jul 2008 07:20:09 GMT

>mobidyc: don't forget to add the -v flag

My 2.5.1 doesn't handle -v with either -C or -A/-B.