topic Re: Parsing a single file into multiple files in Operating System - Linux

Parsing a single file into multiple files

john guardian — Thu, 28 Dec 2006 15:02:45 GMT

I have a file . It has multiple patch listings within, for example line 1 might start out as "s700 10.10". Line 33 (the line number will vary with every new patch file) might start out as "s700 10.26". This file contains "listings" of files for multiple HPUX Operating Systems. I'd like to break it up so that each individual file contains only the list of patches for a particular level of OS (like s700 10.10). Can this be done w/CSPLIT or do I have to use something else? Thanks.

Re: Parsing a single file into multiple files

Sandman! — Thu, 28 Dec 2006 15:08:31 GMT

Imho it would help if you could you provide a sample of the input as well as the desired output.

Re: Parsing a single file into multiple files

john guardian — Thu, 28 Dec 2006 15:24:57 GMT

No problem. Here it is...........
If you view the contents, it's a simple format. I just want to break it up into patch lists for each individual OS. Thanks.

Re: Parsing a single file into multiple files

James R. Ferguson — Thu, 28 Dec 2006 15:31:24 GMT

Hi John:

Indeed, why not use 'csplit'. Given this file:

# cat ./patches
s700 10.10
PHCO_123
PHCO_456
s700 10.20
PHNE_12345
PHNE_22222
s800 11.0
PHKL_18543
PHSS_1234
PHSS_2222

...Using:

# csplit -s -f patchesfor ./patches '/s700 10.10/' '/s700 10.20/' '/s800 11.0/'

...produces files named "patchesfor00", "patchesfor01", "patchesfor02" and "patchesfor03". The first file is empty since there is nothing preceeding the first regular expression argument in the input file.

Regards!

...JRF...

Re: Parsing a single file into multiple files

Sandman! — Thu, 28 Dec 2006 16:04:35 GMT

How about using an awk construct which puts an OSes records into one file:

# awk '{print $0 >> "patch."$3}' file

~cheers

Re: Parsing a single file into multiple files

James R. Ferguson — Thu, 28 Dec 2006 19:39:03 GMT

Hi (again) John:

It seems that a generalized 'csplit' something like:

# csplit -s -f patchesfor patches.txt '/^s/' '/^$/' '{35}'

...which should create files named 'patchesfor00'... dividing them into groups that begin with "s" [for s700 or s800] and ending with a blank line as the file you provide has, should work.

*However*, on my servers, I get an "{35} - out of range" error. You have thirty-one (31) files to generate given your data.

Sandman's suggestion is elegant (as his 'awk' always is!), but with your data, it doesn't behave as well as it should.

For example, within your stanza for s800 11.23 you have variously "11.23" and "11.X". I would think that you would still like to keep these grouped within the same output file.

*Hence*, I'd deploy a Perl solution like this:

# cat ./filter
#!/usr/bin/perl
use strict;
use warnings;
my $fh;
while (<>) {
if (/^s/) {
my @f = split( /[:\s]/, $_ );
my $file = "$f[0]_$f[1]";
open( $fh, ">>", $file ) or die "Can't open $file: $!\n";
}
print $fh $_;
}
1;

...Run as:

# ./filter patches.txt

Your output files will be produced in the directory in which you run this script. The files will be named like "s700_20.20", "s800_11.23", etc. based upon the first two fields (s700 or s800 and the release ID following it).

Regards!

...JRF...

Re: Parsing a single file into multiple files

Gilles Allard — Fri, 29 Dec 2006 09:44:29 GMT

Simple needs require simple solutions. I prefer the following shell script:

FILE=/dev/null
while read LINE; do
if [ -z "${LINE}" ]; then continue; fi
if [ -z "${LINE##s*}" ]; then
FILE=`echo $LINE|tr -s " " "_"`
else
echo $LINE >> $FILE
fi
done
Blank lines will be stripped.
Filenames will be as in JRF post

Re: Parsing a single file into multiple files

James R. Ferguson — Fri, 29 Dec 2006 09:49:01 GMT

Hi John:

By the way, would you please evaluate the responses you received in an earlier post, too. Thanks!

https://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1063457

Regards!

...JRF...

Re: Parsing a single file into multiple files

john guardian — Fri, 29 Dec 2006 10:08:47 GMT

I eval'd the earlier responses early yesterday. If they weren't updated, I'll have to redo. I assigned points and then closed that thread yesterday. I 'll recheck, though. Thanks.

Re: Parsing a single file into multiple files

john guardian — Fri, 29 Dec 2006 10:51:57 GMT

Thanks to everyone for the input. They all worked fine. The scipt is now working well. Regards.

BTW, if anyone hasn't rx'd points, let me know. I believe all w/script suggestions ahve been assigned. Thanks again.

Re: Parsing a single file into multiple files

James R. Ferguson — Fri, 29 Dec 2006 10:59:56 GMT

Hi John:

The thread I would like you to look at dates from September:

https://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1063457

Happy New Year!

Regards!

...JRF...

Re: Parsing a single file into multiple files

Sandman! — Fri, 29 Dec 2006 12:27:01 GMT

On closer inspection I agree with JRF that the awk construct I posted earlier will not do the job owing to the variable format of field three in the input file. Besides I used a small portion of the sample input you had provided to test my script. So I used the entire input file provided, built another script that provides the desired output. And thank you for those words JRF, coming from the master himself :)

Give the awk script below a try:

# awk -F" +" '{if($0!~"^ " && $0!~"^$")p=z[split($2,z,":")-1];print $0>>"patch."p}' file

~cheers

Re: Parsing a single file into multiple files

john guardian — Fri, 29 Dec 2006 13:34:41 GMT

Thanks guys. All input appreciated. I'm off and running now. See you all in the New Year!