Need help with scripts to save contents of logfiles

[Kenneth_18]

Script1 when executed outputs to the screen the contents of a binary logfile composed of tens of thousand of lines.

I want to be able to save the textual output of Script1 to a file for every 1000 lines whereas the first 1000 line output is save to file1 and the next 1000 lines out is save to file2 and so on.

I want to achive this without saving the output of Script1 to a file first and later on be processed by another script.

Can this be achieved with awk or sed? Can someone please start me off or point me in the right direction by providing a sample script that will do the task.

Thanks in advance.

[Steve Steel · Accepted solution]

Hi

Use


split(1) split(1)

NAME
split - split a file into pieces

SYNOPSIS
split [-l line_count] [-a suffix_length] [file [name]]

split [-b n[k|m]] [-a suffix_length] [file [name]]

Obsolescent
split [-n] [file [name]]

DESCRIPTION
split reads file and writes it in pieces (default 1000 lines) onto a
set of output files. The name of the first output file is name with
aa appended, and so on lexicographically, up to zz (only ASCII letters
are used, a maximum of 676 files). If no output name is given, x is
the default.



Steve Steel

[Sanjay Kumar Suri]

split -l 1000 file_name

can do the job.

sks

[Chris Wilshaw]

Split is the command you need, although as you state this is script output you're checking rather than a flat file, use

your_script | split -l 1000

If you want to specify filenames for your output, rather than getting the default xaa, xab etc

use

your_script | split -l 1000 - file_prefix

(note the - to indicate use of standard input as the input file name). The file prefix can be anything you like, for example, if you put the prefix as "logfile", you'd get files of logfileaa, logfileab etc.

[Kenneth_18]

Thanks for the fasr response. I learned a new command again. Is there a possibility for the split command or is their any variant that can have the file number suffix in numerics rather than alphabeth characters?

[Sanjay Kumar Suri]

You can following command:

split -l 10 big Kenneth

And following files will be created:

Kennethaa Kennethad Kennethag Kennethaj
Kennethab Kennethae Kennethah Kennethak
Kennethac Kennethaf Kennethai Kennethal

(big has 112 lines)

sks

[Sanjay Kumar Suri]

You can give the following command:

split -l 10 big Kenneth

And following files will be created:

Kennethaa Kennethad Kennethag Kennethaj
Kennethab Kennethae Kennethah Kennethak
Kennethac Kennethaf Kennethai Kennethal

(big has 112 lines)

sks

[Hein van den Heuvel]

If you want full control over the file name creation then you'll need a little script like the following perl 'one-liner'

your_tool | perl -ne 'if (!($i++%1000)){$x=sprintf(">test_%03d",$n++); open(X,$x)} print X'

Hein.

[Kenneth_18]

Thanks guys for pointing the direction. I found exactly what I'm looking for in "csplit" and the file naming suffix is in numerics.