topic Re: Extracting data in between strings...using perl in Operating System - Linux

Extracting data in between strings...using perl

jmckinzie — Thu, 03 Jan 2008 19:20:12 GMT

I have a variable called WebJump in a perl script.

When i print the content it gives me this:

T
his page is used to hold your data while you are being authorized for your reque
st.

You will be forwarded to continue the authorization process. If this
does not happen automatically, please click the Continue button below.

I want to print everthing between START and END and then reassign it to my WebJump variable.

Any ideas?

Re: Extracting data in between strings...using perl

Tim Nelson — Thu, 03 Jan 2008 19:41:47 GMT

This one is actually an example in the book.

Use sed.

WebJump=`echo "the_stuff"|sed /^BEGIN/,/^END/p`

Re: Extracting data in between strings...using perl

jmckinzie — Thu, 03 Jan 2008 19:44:23 GMT

How can i do it in PERL?

Re: Extracting data in between strings...using perl

Tim Nelson — Thu, 03 Jan 2008 19:45:49 GMT

Alright, almost easy, cept the START is really STARThttp.... and also the END...

Will let you know if I find something. SOrry for the quick wrong answer.

Re: Extracting data in between strings...using perl

jmckinzie — Thu, 03 Jan 2008 19:47:38 GMT

I can change the START END to anything in need to.

IE

...whatever.....

I simply used search and replace....

Re: Extracting data in between strings...using perl

Geoff Wild — Thu, 03 Jan 2008 19:53:20 GMT

Wait for Hein or Merijn to post their fabulous perl answers :)

Re: Extracting data in between strings...using perl

Murat SULUHAN — Thu, 03 Jan 2008 19:54:31 GMT

Hi

If whole text is single line (otherwise please parse into single line)

use index function to find START
use substr function to extract from START to end of the line (str1)

Then you have 2 END in text,
so if you want to extract START and first END
again use index function with str1 to find first END's position. and use substr

if you want to extract BEGIN and second END
use rindex with str1 to find position of second END, then subtract it from len($str1)
use again substr

I have no perl interpreter on this computer so I hope this algorithm will help to you

Best Regards
Murat

Re: Extracting data in between strings...using perl

James R. Ferguson — Thu, 03 Jan 2008 21:50:08 GMT

Hi Jody:

I assume from the content that you truly want everything between the strings START and END, disregarding any newline boundries.

Using a copy-and-paste of your data :

# echo ${WebJump}|perl -000 -nle 'print $1 if m/.*START(.*)END.*/s'

Regards!

...JRF...

Re: Extracting data in between strings...using perl

James R. Ferguson — Thu, 03 Jan 2008 22:30:18 GMT

Hi (again) Jody:

Actually, here is a better representation of what you probably want. It destructively updates the $WebJump scalar variable:

# cat .jody.pl
#!/usr/bin/perl
use strict;
use warnings;

my $WebJump=q[This page is used to hold your da
ta while you are being authorized for your reque st.

You will be forwarded to continue the authorization process. If th
is does not happen automatically, please click the Continue button below.];

( $WebJump ) = ( $WebJump =~ /START(.*)END/s );
print "$WebJump\n";

Regards!

...JRF...

Re: Extracting data in between strings...using perl

Hein van den Heuvel — Fri, 04 Jan 2008 01:06:05 GMT

[Thanks Geoff! I haven't had much time to troll this forum.]

Jody,

Murat pointed out that there are two ENDs in the example. I believe you indicate that was just a bad example, thus nothing to worry about. (allthough it worries me when folks post bad examples, but that's my problem).

Still, if you did have to deal with nested START/ENDs then you may want to be sure to know about the 'greedyness' of the * in perl.

For example if the data was:

x START y START a b d END z END

JRF writes...
( $WebJump ) = ( $WebJump =~ /START(.*)END/s );

This is fine. But note that the .* in there is greedy and in my example will return " y START a b d END z "

If you wanted the inner START/END, then use: .*?

It will return " a b d "

Cheers!
Hein.

Re: Extracting data in between strings...using perl

H.Merijn Brand (procura — Fri, 04 Jan 2008 08:14:48 GMT

What Hein said, but then parsing HTML with regexes is still NOT safe.

Have a look at the HTML::TreeBuilder module.

--8<--- test.pl
#!/pro/bin/perl

use strict;
use warnings;

use HTML::TreeBuilder;

my $tree = HTML::TreeBuilder->new;

my $content = <<'EOH';

This page is used to hold your data while you are being authorized for your
request.

You will be forwarded to continue the authorization process. If
this does not happen automatically, please click the Continue button below.

EOH

$tree->parse_content ($content);
# print $tree->as_HTML (undef, " ", {});

foreach my $f ($tree->look_down (_tag => "form")) {
# print "FORM:\n", $f->as_HTML (undef, " ", {});
$f->as_HTML (undef, " ", {}) =~ m{\bstart(.*?)end}i and
print "START in FORM: $1\n";
}
-->8---

# perl test.pl
START in FORM: https://test.ip.com/siteminderagent/forms/login.fcc?type="33554433&REALMOID=06-00034bb7-e037-116f-8241-808d67a50008&GUID=&SMAUTHREASON=0&METHOD=POST&SMAGENTNAME=$SM$8OJVwItP%2fV8GXRhL%2fhch6KJt3EvC2AWLQ7%2bWLfTgx3%2bWD7k%2buJc3dVSFPOr1jTxg&TARGET=$SM$%2f

Enjoy, Have FUN! H.Merijn [ who does not think this is clean HTML ]

Re: Extracting data in between strings...using perl

jmckinzie — Fri, 25 Jan 2008 02:39:50 GMT

I wound up using regular expressions butam looking forwardto messing withthe treebuilder but it is very very difficult.