topic Re: Perl Data Processing in Operating System - HP-UX

Perl Data Processing

Hakki Aydin Ucar — Wed, 02 Jun 2010 10:28:39 GMT

I want to have a script and read a data file and collects some fields and adds some values . Data file has a header row on top, so I need to get rid of this non-numeric part of file during read - write processing. Which method do you recommend to achieve this ?

my data file looks like; (delimited with ",")
Date,Time,Name,
05-27-2010,00:30,Isbl

Re: Perl Data Processing

James R. Ferguson — Wed, 02 Jun 2010 10:43:15 GMT

Hi:

Other than skipping the header and potentially any lines that don't begin with a number in the first field, your request is sparse.

That said, you could do:

# perl -ne 'print if /^\s*\d/' file

...or:

# perl -ne 'next unless /^\s*\d/;print' file

...or:

# perl -e 'while (<>) {next unless /^\s*\d/;print}'

Regards!

...JRF...

Re: Perl Data Processing

Hakki Aydin Ucar — Wed, 02 Jun 2010 12:56:57 GMT

Hello James,
This is good, do I have to call it out of my script ? OR can I embed to my code ? How can I embet this code clause ?

Re: Perl Data Processing

James R. Ferguson — Wed, 02 Jun 2010 13:11:08 GMT

Hi (again):

This is good, do I have to call it out of my script ? OR can I embed to my code ? How can I embet this code clause ?

In a shell program you can simply in-line the code like:

...
echo "processing ${FILE}"
perl -ne 'print if /^\s*\d/' ${FILE}
...

If you want more than a commandline script we can create a "full" Perl script that looks just like any command or shell script that you would otherwise call from a shell.

Regards!

...JRF...

Re: Perl Data Processing

H.Merijn Brand (procura — Wed, 02 Jun 2010 15:26:59 GMT

If your data is delimited by comma's, and has a header, I expect it might be CSV data :)

You ought to take a look at Text::CSV (or Text::CSV_XS for faster processing);

$ cpan Text::CSV_XS

$ perldoc Text::CSV_XS

Enjoy, Have FUN! H.Merijn

Re: Perl Data Processing

Hakki Aydin Ucar — Thu, 03 Jun 2010 06:41:16 GMT

>Merijin:
You ought to take a look at Text::CSV (or Text::CSV_XS for faster processing);

Hi , is this the right doc. you recommended?
http://search.cpan.org/dist/Text-CSV_XS/

Regards,
Hakki

Re: Perl Data Processing

James R. Ferguson — Thu, 03 Jun 2010 11:46:01 GMT

Hi (again) Hakki:

The link to Text::CSV_XS that Merijn posted will also show you the documentation. You can also use:

http://search.cpan.org/dist/Text-CSV_XS/CSV_XS.pm

By using his module you avoid beginning to slide down a slippery slope as first your comma-delimited data begins to contain quoted fields with embedded commas to separate subfields :-) Handling cases like that would cause you to re-invent what already works!

Regards!

...JRF...