script help

[Deepak Seth_1]

HI Everybody,

I have attached a text file . The text file contains the information of the different changes we made to our systems in some format.

I would like to sort the text file by "Time/Date+system name" - which is 1st and 3rd field line containing date/time stamp(without impacting anything on the text written under it) . Pls. refer to the attached text file for what i would like to see after sorting.


thanx

[harry d brown jr]

Deepak Seth,

Start by splitting the file:

csplit -f dumb test.txt '/____________________________ Reply Separator _______________________________/1' {*}

then sort the files (PREFIXED with "dumb") based upon date:

grep "^Date:" dumb[0-9]* | tr -s " " " " | sed "s/:/+/" | sed "s_/99 _/1999 _" | sed "s/\( [1-9]\:[0-9][0-9]\)/ 0&/" | sed "s/ 0 / 0/"| sort -t"+" -k 2 | cut -d"+" -f1 | xargs -i cat {} >>newfilename

live free or die
harry
harry

[harry d brown jr]

Deepak Seth,

Nothing like seeing my name twice!

The statement:

tr -s " " " "

Has TWO spaces in the first set ouf quotes "ss", ant the second has one space "s", as in:

tr -s "ss" "s"

live free or die
harry

[Deepak Seth_1]

I think i attached the wrong file. Iam really sorry if i wasted your time.


Here the output of the text file in case u have problem reading the attached file.

2002:03:26:13:50 root bvapp1
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:50 root bvapp2
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:51 root devhp1
created file systems for baandcdg - 25 Nos. (24 - database mount
points and 1 - application)

2002:03:26:14:55 root bvapp1
change dbc_max_pct from 50% to 25%.



After sort it should be :

2002:03:26:14:55 root bvapp1
change dbc_max_pct from 50% to 25%.

2002:03:26:13:50 root bvapp1
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:50 root bvapp2
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:51 root devhp1
created file systems for baandcdg - 25 Nos. (24 - database mount
points and 1 - application)

[harry d brown jr]

SAME CONCEPT:



csplit -f dumb test.txt '/^[0-9][0-9][0-9][0-9]:/1' {*}

**** test.txt is the file with that stuff in it

then sort the files (PREFIXED with "dumb") based upon date:

grep "^[0-9][0-9][0-9][0-9]" dumb* | sort | cut -d":" -f1 | xargs -i cat {} >>newfilename



live free or die
harry Deepak Seth

[harry d brown jr]

crap that won't work

and what a bad paste job I did!!!

Deepak, is the "blank" line blank, meaning that it only contains a Carriage Return?


live free or die
harry

[harry d brown jr]

Deepak,

Try this:

csplit -f dumb test.txt '/^$/1' {*}

**** test.txt is the file with that stuff in it

then sort the files (PREFIXED with "dumb") based upon date:

grep "^[0-9][0-9][0-9][0-9]" dumb* | sort | cut -d":" -f1 | xargs -i cat {} >>newfilename


live free or die
harry

and yes "harry deepak seth" was the bad paste job!!! copied three post in a row - dumb!

[SHABU KHAN]

Deepak,

This little script will sort the first field (date/time) and will keep the remaining text:
#!/bin/ksh

rm -f /tmp/test.out
MY_PATTERN="[0-9][0-9][0-9][0-9]:[0-9][0-9]:[0-9][0-9]:[0-9][0-9]:[0-9][0-9]"
grep "${MY_PATTERN}" test.in | sort -r -k1 > /tmp/sorted
CNT=0
cat test.in | while read LINE; do
FOUND=`echo ${LINE} | grep "${MY_PATTERN}"`
if [[ -n "${FOUND}" ]]; then
CNT=`expr ${CNT} + 1`
head -"${CNT}" /tmp/sorted | tail -1 >> /tmp/test.out
else
echo "${LINE}" >> /tmp/test.out
fi
done

Your input file is test.in and the output you need will be in test.out..

-Thanks,
-Shabu

[Deepak Seth_1]

You are my man of day - almost . I try with test.txt which is sample file and it works like a magic .
But when i try to do this with a bigger file(55 K) , i ran into problem with this error - reached limit on number of output files at arg {*}

We are almost there. Any other suggestion .

[Deepak Seth_1]

The last message for Harry. Iam trying Sabhu script now. will revert u shortly on the outcome of Sabhu script. thanx guys.

[harry d brown jr · Accepted solution]

Deepak,

add this option to csplit:


-n 8

it will make the DIGITS after the "dumb" prefix 8 digits long, should be enough!

live free or die
harry

[harry d brown jr]

Shabu,

BUT, if you had two or more blocks with the same DATA header, it would only match the first one, losing the second or more, by duplicating the first. Try this:


2002:03:26:13:50 root bvapp1
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:50 root bvapp1
some other message. that would or could be missed.
because of the grep. awk could do it, provided you "trimed" out the text in the original $line.


live free or die
harry

[Deepak Seth_1]

You are my man and so is sabhu khan. Your effort provided me a quick solution.

thank your very much .

[SHABU KHAN]

Deepak,
You are Welcome, good to know that our postings helped.

Harry,
Actually my script will work even if it has the same DATA header. I tried it with your example and it works..

Thanks,
Shabu

[SHABU KHAN]

Deepak,
You are Welcome, good to know that our postings helped.

Harry,
Actually my script will work even if it has the same DATA header. I tried it with your example and it works..

Thanks,
Shabu

[harry d brown jr]

Shabu,

I used this as an input file:
-------------------------
# more test.in
2002:03:26:13:50 root bvapp1
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:50 root bvapp1
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:50 root bvapp2
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:51 root devhp1
created file systems for baandcdg - 25 Nos. (24 - database mount
points and 1 - application)

2002:03:26:14:55 root bvapp1
change dbc_max_pct from 50% to 25%.

2002:03:26:13:50 root bvapp1
some other message. that would or could be missed.
because of the grep. awk could do it, provided you "trimed" out the text in the
original $line.
-------------------------
and ran your script, and this is what I got:
-------------------------
# more test.out
2002:03:26:14:55 root bvapp1
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:51 root devhp1
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:50 root bvapp2
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:50 root bvapp1
created file systems for baandcdg - 25 Nos. (24 - database mount
points and 1 - application)

2002:03:26:13:50 root bvapp1
change dbc_max_pct from 50% to 25%.

2002:03:26:13:50 root bvapp1
some other message. that would or could be missed.
because of the grep. awk could do it, provided you "trimed" out the text in the
original $line.
-------------------------

Which only "sorted" the headers and did not move the text with it. When it should look like this:
-------------------------
# more test.outnew
2002:03:26:13:50 root bvapp1
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:50 root bvapp1
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:50 root bvapp1
some other message. that would or could be missed.
because of the grep. awk could do it, provided you "trimed" out the text in the
original $line.

2002:03:26:13:50 root bvapp2
unc-commented cronjobs related to BV . Srini wanted them to be
un-commented after they successfully migrated for Northern Europe.

2002:03:26:13:51 root devhp1
created file systems for baandcdg - 25 Nos. (24 - database mount
points and 1 - application)

2002:03:26:14:55 root bvapp1
change dbc_max_pct from 50% to 25%.
-------------------------
Which requires a change to my sort (my excuse last night - I couldn't VPN into work):

sort -t: -k2,6

resulting in the last line like this:

grep "^[0-9][0-9][0-9][0-9]:[0-9][0-9]:[0-9][0-9]:[0-9][0-9]:[0-9][0-9] " dumb[0-9]*
| sort -t: -k2,6 | cut -d":" -f1 | xargs -i cat {} >>test.outnew

live free or die
harry

[SHABU KHAN]

Oops ! Bummer !

I see what you are saying ... I do not deserve those points after all..

I was trying to do this programmatically without splitting the files ... hmm..

Thanks Harry for pointing this out ...

-Shabu

[Deepak Seth_1]

Don't worry about the points . I think i was very happy with the solution of harry and it seems have not paid close attention your script and its output.

I can't minus your point so treat them as advance payment for my next question - someday !!!!!

thanx