Operating System - HP-UX
1836878 Members
2371 Online
110111 Solutions
New Discussion

Re: how to divide big file into several small files?

 
youbojun
Occasional Contributor

how to divide big file into several small files?

We installed domino5.09 on HP unix 11.0. The log file of Domino server increased very quickly. Now I have one log file which size has more than 1G. So I can't open it with word. Can you tell me how to divide the big file into several small files? or how to open it?
Everyone must be kind each other.
13 REPLIES 13
Rajeev  Shukla
Honored Contributor

Re: how to divide big file into several small files?

Hi,
Go through the man pages of "tail".
Use this command to take the respected lines of the logfile. Like use
tail -100 and put them to a file this way you can trim the file to small pieces.
Or use sed to save the lines to some file
sed '100q' logfile >log1

or
sed -n 20,50p logfile >log2
prints 50 lines after 20th line to a file.

Cheers
Rajeev
Robert-Jan Goossens
Honored Contributor

Re: how to divide big file into several small files?

Hi,

You can use split take a look at the man page.

Robert-Jan.
Vincent Fleming
Honored Contributor

Re: how to divide big file into several small files?

As Robert-Jan suggested, the "split" command will split it into multiple parts.

If you want to view it without breaking it up, try the "more" command. You can search for things with "more" also - such as dates or times in the log:

more logfile

then type:

/Jan 6

and it will jump to where "Jan 6" appears first in the file. The spacebar will move ahead one page.

Type "man more" for more information.

Good luck,

Vince
No matter where you go, there you are.
Bill Hassell
Honored Contributor

Re: how to divide big file into several small files?

Like most log files, they grow without bopunds and the first task in managing disk space is to use cron to trim these files. That said, a 1Gb logfile is way out of control. How long did it take to grow thayt large? Is it full of repeated messages? Is there a setting in Domino to control the verboseness of this logfile?

In trimming any logfile, you need to look at example contents to determine what is important to save, A logfile trimmer script might scan for certain keywords, extract those entries into an archive and then zero the file length. Or a trimmer might copy the file weekly (daily?) into a series of files and use compress to save space. As the admin, you need to determine how long too keep the files. I prefer to keep a quantity (5 or 10) and trim the logfiles only when they exceed a certain size, say 1 to 5 megs.

For verbose logs that have a lot of generally useless info, scanning and saving the important messages makes more sense. Here's a quick line that works well for syslog.log:

grep -i -e err -e warn -e fail -e alert -e fatal -e crit /var/adm/syslog/syslog.log



Bill Hassell, sysadmin
KVS Raju
Advisor

Re: how to divide big file into several small files?

Hi,

Using split command you can split the files,

example:

split -b nk

split command split into pieces n*1024 bytes in sizes.

See the man page of split.

Regards
Kvs Raju


Time and Tide wait for none
Trond Haugen
Honored Contributor

Re: how to divide big file into several small files?

Pharao Bill has a very good point. If there are messages of importance in that logfile. they should be read and actet on before they get too old.

Regards,
Trond
Regards,
Trond Haugen
LinkedIn
Adam J Markiewicz
Trusted Contributor

Re: how to divide big file into several small files?

Hi

tail is usefull for real-time tracing, but don't use it with big files, as it has buffer limited to 20kB only.

Split will create you a copy of everything in several files. If you want only interesting part (from byte no. to byte no.) you can use dd.

Good luck
Adam Markiewicz
I do everything perfectly, except from my mistakes
Pete Randall
Outstanding Contributor

Re: how to divide big file into several small files?

Quote:
"So I can't open it with word"
unquote

Is this a Unix system?



Pete

Pete
Pete Randall
Outstanding Contributor

Re: how to divide big file into several small files?

Quote:
"So I can't open it with word"
unquote

Is this a Unix system?

What I mean is: is this a Unix system that you're trying to deal with the log file on? If you're trying to do this from a Microsoft PC, it's going to be difficult. You need to log on to the HP system itself. The you can use tail and split and all the other suggestions above.



Pete

Pete
Chris Vail
Honored Contributor

Re: how to divide big file into several small files?

I have attached the documentation for a script I wrote that is used every day by all 50+ Unix systems we have here. This script parses a log file (in HP-UX, its syslog.log, but you can change this). In parsing, it removes redundant/uninteresting data by using a filter file. This file contains data that is expected and normal for that particular computer, and therefore uninteresting on a daily basis.
You will probably have to adapt the script to your own format. Its set up to grep the log file by date. I don't know how domino sets up its log file(s), but at the very minimum, you can adapt the script for your own use.
For example: this morning in my in box I got the filtered system log from the NTP server. This recorded almost 13,000 entries yesterday--most of them NTP entries, but a few FTP entries as well. The filtering process removed 12,967 entries, leaving only about 30 lines to look at.
At the end, it emails the results of the filtering process to me and to others.
Jose Mosquera
Honored Contributor

Re: how to divide big file into several small files?

Hi,

The split command will be useful for you, pls check "man split" to a get a lot of options about. Also sed command will be a possiblity:

sed -n ", p" >

Rgds.
MANOJ SRIVASTAVA
Honored Contributor

Re: how to divide big file into several small files?

do a man split ,

split splits the file in to half or as per the lines specified.


Manoj Srivastava
Nick Wickens
Respected Contributor

Re: how to divide big file into several small files?

The alternate csplit is also handy if you want to split at each instance of an expression.
Hats ? We don't need no stinkin' hats !!