Sorting data in a big file

[Stuart Powell]

I create a rather large file every day, that I would like to break down in to numerous little files. The file contains process data. Each line of the file has 101 fields seperated by commas ","; 100 data fields, 1 date time field.
I have another file, with a number of lines, that defines the production batches. Each line in this file consist of the three fields; the batch name, the start date and time for the batch, and the finish date and time for the batch.
I have created a shell script (attached) that attempts to lay the foundation for looping through all of the batches in the second file, and create a seperate file for each batch from the data saved in the first file. However, when I execute this script I always get the following error from sed: "sed: Function /Mar cannot be parsed."
I have not used sed before, so I realize that it may not be the best solution. I am open to other ideas.
I'm working on a HP-UX 10.20, 700 series machine.

[Andreas Voss · Accepted solution]

Hi,

possibliy the sed confuses because you haven't quoted the expression. Try this:

sed -e "${starttime},${dofftime}p" < $file1 >$mrnum

Regards

[Stuart Powell]

That fixed the sed error. I get the new file correction working completely I had to change the -e to a -n. When I did that the script created the correct file.
Thank you

[Luc gosselin]

I don't know at what level you are looking for but for a quick and dirty sort I use

"sort filename|uniq -d"

You can pipe it to a file or just the way it is to sort the existing file. Hope it helps...