While Loop Question?

[Allanm]

I have a file which has the following data -

emailid1 word1 word2 word3
emailid2 word1 word3

The spaces between the words are tab separated spaces.

I want to echo each word on the line separately and at the same time consider the next line as a new line.

Please help!

[Michael Steele_2 · Accepted solution]

cat file | while read a b c
do
echo $a $b $c
done

[Allanm]

Thanks Micheal!

There is one more piece to it, the words can be more than 3 and the count cannot be predicted. I need a more dynamic way.

[Allanm]

And dont want to print blank lines.

[James R. Ferguson]

Hi Allan:

while read LINE
do
echo ${LINE}|tr -cs "[:alpha:]" "[\012*]"
done < file

Spaces and/or tabs can separate the words in the file.

Regards!

...JRF...

[James R. Ferguson]

Hi (again) Allan:

> And dont want to print blank lines.

OK, so use:

# cat ./lister
#!/usr/bin/sh
while read LINE
do
[ -z "${LINE}" ] && continue
echo "${LINE}"|tr -cs "[:alpha:]" "[\012*]"
done < $1

...run as:

# ./lister file

Regards!

...JRF...

[James R. Ferguson]

Hi (again):

In keeping with TIMTOWDI here's another way:

# perl -nle 'next if m/^\s*$/;print join "\n",split' file

Regards!

...JRF...

[Allanm]

Thanks JRF!

But this doesn't do what I want to achieve -

Email-ids are of the pattern foo_bar@foobar.com and it is the first entry on each line.I want to print that as is and add a certain html tags around it -



and for the words like words1... I need word1
word2
...

The final document should look like the following -



word1
words2
words3




word1
words3

[Dennis Handly]

>I want to echo each word on the line separately

for word in $(< file); do
echo $word
done

[Hein van den Heuvel]

Hmm,

Seems like your original problem statement can be satisfied with a simple:

$ tr "\t" "\n" < tmp.txt

look:

$ cat tmp.txt
asergfewgfwegfew aap noot mies
blah more data
$ od -c tmp.txt
0000000 a s e r g f e w g f w e g f e w
0000020 \t a a p \t n o o t \t m i e s \n b
0000040 l a h \t m o r e \t d a t a \n

$ tr "\t" "\n" < tmp.txt
asergfewgfwegfew
aap
noot
mies
blah
more
data

But that's not really what you want apparently, and it looses the 'specialness' of the first word. Why not do it all in one perl one-line or program?


$ perl -ne 'chomp; @words=split; $u=shift @words; print "\n\n"; for (@words) {print "$_<
/word>\n"}; print "\n\n"' tmp.txt


aap
noot
mies




more
data



Enjoy!
Hein

[Laurent Menase]

while read a b
do
set -- $b
echo 'email $a'
while [ $# != 0 ]
do
echo word $1
shift
done
done

[James R. Ferguson]

Hi Allan:

Your original query was "I want to echo each word on the line". Then, after a couple of offerings, you wrote, "Email-ids are of the pattern foo_bar@foobar.com and it is the first entry on each line.I want to print that as is and add a certain html tags around it -".

This whole question, beginning with the subject was poorly framed. The use of the word "word" in a language/scripting context is loaded with meaning alone. You offered your input. Offering a _sample_ of the output you wanted might have gone a long way to a more direct, faster solution.

Regards!

...JRF...

[Allanm]

Thanks Hein,

Works as expected, a couple of conditions where it can fail or I would like to handle is if there is a blank line or a line which is commented with a hash (#) then I would not like it to print anything and go on to the next entry.

Thanks,
Allan.

[James R. Ferguson]

Hi Allan:

> a couple of conditions where it can fail or I would like to handle is if there is a blank line or a line which is commented with a hash (#) then I would not like it to print anything and go on to the next entry.

See if this meets your needs:

# cat ./myfilter
#!/usr/bin/perl
use strict;
use warnings;
my ( @F, $name, $word );
while (<>) {
s/([^#]*).*/$1/; #...snip off "#" and any trailing chars
next if m/^\s*$/; #...skip if only whitespace
@F = split;
$name = shift @F;
print qq(\n\n);
for $word (@F) {
print qq(), $word, qq(\n);
}
print qq(\n\n);
}
1;

Now consider an input file like this:

# cat -etv ./myinput
emailid1 word1^Iword2 word3$
$
# this is a comment line$
^I$
emailid2 ^I worda wordb # and this piece is commentary too$

...as you can see the ^I are tab characters and the $ characters mark the end-of-line. Hence there are blank lines and comments signaled by hash ("#") marks at various points.

Now:

# ./myfilter ./myinput


word1
word2
word3




worda
wordb



Regards!

...JRF...

[Allanm]

Thanks JRF/Hein.

The only change I made was to -

s/#.*//;

instead of

s/([^#]*).*/$1/

=============

Another set questions come to my mind since we are doing this in Perl.Originally I was working on shell script for this and the following variables were set related to saving the original file and saving the output in a intermediate XML file and moving that file over to be the destination file -

Originally when I was planning this to do this in shell I was having something similar -

===============================

env=`hostname |awk -F"." '{print $2}'`
destination_xml="/a/g/m/admin.xml"
DATE=`date '+%Y%m%H%M'`
Intertim_xml="/a/g/m/admin.xml.interim.$DATE"

if [ $# -lt 1 ]
then
echo
echo "Correct USAGE:"
echo -e "\t\t$0 \n"
exit
fi

# Save the target XML file before renewing it
cp ${destination_xml} ${destination_xml}.${DATE}


while read Line {
....
} 1>> $Interim_xml

mv ${Interim_xml} ${destination_xml}

====================

How do I port this in Perl?

Thanks,
Allan.

[James R. Ferguson]

Hi (again) Allan:

> Another set questions come to my mind since we are doing this in Perl. Originally I was working on shell script for this and the following variables were set related to saving the original file and saving the output in a intermediate XML file and moving that file over to be the destination file -

This is painless in Perl (of course!). Simply add the '-i' switch to enable inplace editting.

Using my last script, change:

#!/usr/bin/perl

to:

#!/usr/bin/perl -i.old

This specifies an "in-place" edit of the input file (passed as the argument to the script). A backup copy of the original file is made automatically, nameing it with its original name suffixed with ".old" [or whatever you want].

Regards!

...JRF...

[Allanm]

as always Thanks JRF!!! :)

That takes care of the input file.

I want that to happen to the interim file and as well as target xml file. (as described in the "cp" and "mv" commands above).

How do I achieve that?

How do I specify the following variables in the Perl script -

env=`hostname |awk -F"." '{print $2}'` # so that the script run in any environment
destination_xml="/a/g/m/admin.xml" #target xml
DATE=`date '+%Y%m%H%M'`
Intertim_xml="/a/g/m/admin.xml.interim.$DATE"

Also I would like to handle a failure condition wherein if the input file is empty then it should use the previously saved input file...

[Hein van den Heuvel]

>> I want that to happen to the interim file and as well as target xml file. (as described in the "cp" and "mv" commands above).


You want to try to use the perl command "rename"
Ir you can call: system(qq(cp.... ))


>> How do I specify the following variables in the Perl script -
env=`hostname |awk -F"." '{print $2}'` # so that the script run in any environment

There is a perl module for that:
Try this: perl -e "use Sys::Hostname; print hostname"

So just use the function 'hostname' in the program, after declaring it from "Sys"

>> destination_xml="/a/g/m/admin.xml" #target xml

Just a piece of string.

>> DATE=`date '+%Y%m%H%M'`

There is a locasltim function for that:

perl -e '($sec,$min,$hour,$mday,$mon,$year)=localtime; printf qq(%d%02d%02d%02d\n),$year+1900,$mon+1,$day,$hour,$min'


>> Also I would like to handle a failure condition wherein if the input file is empty then it should use the previously saved input file...

You could protect that with some defensive shell code before calling perl.
Of, in the perl script you couls explicitly create the output only if data exists.

Tou can pre-test with the '-s ' function,
or you can loop and only open on first data"

While (<>) {
open $out,">x.x" unless $out
:
}


Good luck!
Hein

[Allanm]

Thanks Hein,

I did a small mistake, in the destination xml I want to use the env variable so that it can run on different environments and be environment independent.


env=`hostname |awk -F"." '{print $2}'`
destination_xml="/a/g/$env/admin.xml"

How do I do that in Perl script.

Thanks,
Allan

[James R. Ferguson]

Hi Allan:

> env=`hostname |awk -F"." '{print $2}'`
> destination_xml="/a/g/$env/admin.xml"
> How do I do that in Perl script.

As Hein said, there is a Perl module to fetch the hostname:

#!/usr/bin/perl
use strict;
use warnings;
use Sys::Hostname;
my $env=hostname;
my $destination_xml = "/a/g/$env/admin.xml";
print "I point to '$destination_xml'\n";

That said, I am a bit confused when you said, "Originally I was working on shell script for this and the following variables were set related to saving the original file and saving the output in a intermediate XML file and moving that file over to be the destination file".

Specifically, what's wrong with the inplace edit I showed you? After all, Perl does what you would do via an intermediate file with a Perl-slight-of-hand. If the updated file isn't what you want, the use of the ".old" [or whatever you would choose] allows you to restore the original file by 'rename'ing it as Hein noted.

Regards!

...JRF...

[James R. Ferguson]

Hi Allan:

> in the destination xml I want to use the env variable so that it can run on different environments and be environment independent.

If you truly mean any variable in your environment as 'env' in a shell would show you, then modify and reference the %ENV hash:

# export NAME=Allan;perl -le 'print "I see $ENV{NAME} using HOME=$ENV{HOME}"'
I see Allan using HOME=/root

Here, I created an environmental variable named "NAME"; assigned the value "Allan" to it; and then reported its value in a perl snippet together with the pre-defined 'HOME' variable.

Regards!

...JRF...

[Allanm]

I think I was misunderstood in relation to env variable.

Few of the servers I have , have the hostname like app.prod.imt.ei.foobar.com, out of which I took out prod and saved it as env variable (can be any name).

Since the XML I am saving to has path which has that $env variable in it so wanted to utilize the variable there.

XML Path is - /a/g/$env/admin.xml
So when I run the script on a prod server I get -
/a/g/prod/admin.xml

like in qa environment I would get /a/g/qa/admin.xml

I generally capture that in a shell script using awk -

Env=`hostname |awk -F"." '{print $2}'`

What would be the one-line perl equivalent of this so that I can save it as a variable and use it when defining the path of the xml file.

Thanks,
Allan.

[Allanm]

I achieved extracting the environment out of the hostname. The only thing left is for checking for an existence of an email address - foo@foobar.com on the interim file before overwriting the target xml file.

I want something like this in Perl -

var=`grep "@foobar.com" file |wc -l`
if [$var > 1]

then
mv interim.xml destination.xml
fi

Thanks,
Allan

[Hein van den Heuvel]

Not entirely sure what you want, be here are some observations which may help.

>> var=`grep "@foobar.com" file |wc -l`
if [$var > 1]

Why pipe into 'wc -l' just use `grep -c ...`

But since you do not care how many matches, why not just look at the return status

Just use:

grep -q ...
if [$? == 0 ]


Now a perl program like mine ,here presented as program, not one-liner or or JRF's, is the ideal place to not just look for a that target string anywhere but only where is it meaningful.
It too can return a status, based on having seen a proper Email or not:

-----------------------------
$email_not_seen = 1;
while (<>) {
chomp;
@words=split;
$u=shift @words;
$email_not_seen = 0 if $u =~ /\@foobar.com/;

print "\n\n";
for (@words) {
print "$_\n"
}
print "\n\n"'
}
exit $email_not_seen


Of course perl coudl also do somethinh like.. (untested):

-----------------------------
$xml = 'interim.xml';
open XML ">xml";
while (<>) {
chomp;
@words=split;
$u=shift @words;
$email_seen = ($u =~ /\@foobar.com/);

print XML "\n\n";
for (@words) {
print XML "$_\n"
}
print XML "\n\n"'
}
close XML
if ($email_seen) { rename $xml, 'final.xml' }


Enjoy!
Hein.

[James R. Ferguson]

Hi (again) Allan:

> I want something like this in Perl -
> var=`grep "@foobar.com" file |wc -l`
> if [$var > 1]
> then
> mv interim.xml destination.xml
> fi

There are various ways; one is:

#!/usr/bin/perl
use strict;
use warnings;
my $old = qq(interim.xml);
my $new = qq(destination.xml);
my ( $n, @lines );
open( my $fh, '<', $old ) or die "Can't open '$old': $!\n";
@lines = <$fh>;
$n = grep( /\@foobar.com/, @lines );
if ( $n > 1 ) {
rename $old, $new;
}
1;

Instead of reading the whole file into the '@lines' array, however, you could match and count as you go along since ultimately all you care about is the you have more than one occurance of your match string. This would faster for large files since you could exit the read loop early if/when the triggering count is met. You could do this like:

#!/usr/bin/perl
use strict;
use warnings;
my $old = qq(interim.xml);
my $new = qq(destination.xml);
my $n = 0;
open( my $fh, '<', $old ) or die "Can't open '$old': $!\n";
while (<$fh>) {
$n++ if m/\@foobar.com/;
if ( $n > 1 ) {
close $fh;
rename $old, $new;
last;
}
}
1;

Regards!

...JRF....