Operating System - Linux
1748170 Members
3964 Online
108758 Solutions
New Discussion юеВ

Re: Pull data from file with script tool?

 
SOLVED
Go to solution
TDW
New Member

Re: Pull data from file with script tool?

Thanks for the replies guys! All three of you came up with a viable solution via different methods. They all return these values from my original test data:

474 662
999 662
123 789
515 789

Hein, the last code you gave seems to start running and then hang , like this:

perl -n perlHein.pl /tmp/testdata
999 662
474 662

Not sure why that is happening.

Thanks much for sharing your skill!
Hein van den Heuvel
Honored Contributor

Re: Pull data from file with script tool?

in my first solutions I used "perl -n x.pl x

The x.pl is the perl script text
The x was the data file name
The -n tell perls to create an implied loop to process each input line.

The last solution was presented as a full program, invoking perl itself, and with the loop explicitly coded.
Loop: while (<>) {

That solution should be invoked with:
./script_name file_name
Sorry for not making that clear.
And I forgot to mark the 'retain formattng' option while posting, so the loop was not visibile with the indenting either!

The 'hang' is reading from STDIN for more data to process.

Sorry 'bout that confusion!
Hein.
Sandman!
Honored Contributor

Re: Pull data from file with script tool?

Agree with JRF that there is no need to first sort the file and then pipe it to an external program when the entire procedure can be scripted within awk. That said here is an improved version of the awk construct posted earlier:

# cat file
123 789
234 500
999 662
373 881
474 662
611 200
515 789
809 424
234 500

# awk '{i=$2;if(x[i] && x[i]!=$1) print x[i],i"\n"$0;x[i]=$1}' file
999 662
474 662
123 789
515 789
Hein van den Heuvel
Honored Contributor

Re: Pull data from file with script tool?

Sandman!

Two rules to practice thsi week...

1) As simple as possible, but no simpler
2) leave well enough alone



Your new examples only work for certain data sets. It's much similar to my first solution and fails for much the same reason
It remembers only left value for a given right value, where there might be a list.
And on each 'fresh' value it reprints the save values potentially making fresh dups as it goes. Try this data set...
999 662
474 662
333 662
123 457
474 662
811 363
474 662
123 789
234 500
999 662
373 881
474 662
611 200
515 789
809 424
234 500

Cheers,
Hein.



Sandman!
Honored Contributor

Re: Pull data from file with script tool?

Good point Hein as I found out on closer inspection and with the dataset you supplied. And yes your Perl code is algorithmically similar to the one I wrote in awk.

Each record in the input file is replaced by the succeeding one and since the file is not sorted the dups fall thru the cracks. But this horse ain't dead yet ;) so here is the final version of the awk construct.

awk '{
if (x[$0]!=$0) {
y[$2]=(y[$2]?y[$2]"\n"$0:$0); z[$2]++
} x[$0]=$0
}END{
for (i in y) if (z[i]>1) print y[i]
}' file