- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: How to split a large file
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Forums
Discussions
Discussions
Discussions
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2004 07:13 PM
08-03-2004 07:13 PM
How to split a large file
I'm looking for a way I can split an Oracle alert log based on month. Currently I go into vi, create a new file with the data required, and then purge out of the current alert log. I've currently got an alert log which is about 420Mb in size. I would like to be able to delete all lines upto, say 1/8/04.
Any suggestions?
TIA,
Scott Taylor.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2004 07:22 PM
08-03-2004 07:22 PM
Re: How to split a large file
for splitting a file based on expressions have a look at csplit. This will allow you to split the log file based on Dates in this file.
For deleting up to an date i would use perl. this is flexible and capable of handling large files.
Hope this helps
Regards Stefan
- Tags:
- csplit
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2004 07:24 PM
08-03-2004 07:24 PM
Re: How to split a large file
Find out the line number from where you want to keep the alert log. Then use tail command
tail -nline_number alert_log>new_alert_log
mv alert_log old_alert_log
mv new_alert_log alert_log
sks
- Tags:
- tail
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2004 07:34 PM
08-03-2004 07:34 PM
Re: How to split a large file
You can try 'csplit' with regexpression
You can scan for a regex then delete all lines preceding it ..
csplit
See man csplit
HTH
Regards,
Ram.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2004 07:35 PM
08-03-2004 07:35 PM
Re: How to split a large file
basically you can split up a large file with the equally named "split" command where you can determine the size of the pieces by your own.
In your special case you can use
vi
:set number (show line numbering)
then search the first line you want to keep (eg. 120000)
then type 1G (top of file) and type 120000dd
this will delete the first 120000 lines and let you just the rest you want to keep.
Maybe there are better solutions, but at least it works :-)
Regards
Franky
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2004 07:37 PM
08-03-2004 07:37 PM
Re: How to split a large file
open the log file using vi
:setnu
assign the numbers to each line, check for the lines upto 1/8/04 (note the line number)
spilt -l
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2004 08:33 PM
08-03-2004 08:33 PM
Re: How to split a large file
how to split the alert file by month? Simple: start a cronjob each first or last minute of the month and move the file.
Personally I cut off the first n lines when the alert file exceeds a specified size.
regards,
Thierry.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-03-2004 10:13 PM
08-03-2004 10:13 PM
Re: How to split a large file
To get this month informations,
sed -e
To purge that details from original files,
echo "`sed -e
-- muthu --
- Tags:
- sed
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-04-2004 12:01 AM
08-04-2004 12:01 AM
Re: How to split a large file
for me cat (or more) works:
cat "alert log" | grep "string defining your month" > "new file"
HTH
Volkmar
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-04-2004 12:29 AM
08-04-2004 12:29 AM
Re: How to split a large file
You can use awk as well to do something like this (although awk has a limit to the max fields per line that can sometimes mess you up).
If you post the layout of the file (say a few sample lines) so I can see what the date string looks like, I could probably whip up a script in short order.
Best regards,
Kent M. Ostby
- Tags:
- awk
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-04-2004 02:37 AM
08-04-2004 02:37 AM
Re: How to split a large file
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=654588
Oracle alert files have times stamps like:
Fri Jun 25 14:19:13 2004
I'd use a perl script (let's call it alert-by-month.pl) similar to
if (/^\w+\s+(\w+)\s+\d+\s\d+:/) {
if ($month ne $1) {
$month = $1;
open(M,">alert_$month.log") or die;
}
}
print M;
So it looks for lines starting with a word [the day: \w+] followed by whitespace.
It remembers the next word in $1 [the month: (\w+)]
Then, to be sure it is a date stamp is looks for whitespace, numbers, whitespace, numbers, colon" : [day,hour,":" : \s+\d+\s\d+: ]
When it sees that, it compares the new month (in $1) with the current month.
If it is a new month (like the first line!), then it makes that the current month and opens a new output file.
Finally it prints the line on the current file.
Now exectute with:
perl -n alert-by-month.pl alert
Good luck.
Hein
- Tags:
- Perl
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-04-2004 02:51 AM
08-04-2004 02:51 AM
Re: How to split a large file
Create a script file called "do":
echo MONTH $1 > .do1
cat $2 >> .do1
awk -f pull_month.awk < .do1 > .do3
mv .do2 archived_data.log
mv .do3 new_data.log
rm .do1
Create an awk file called pull month :
/^MONTH/ {targetmonth=$2;next;}
$2==targetmonth {print $0 > ".do2";next}
{print $0}
Run it as:
do Jun
Example:
do Jun usek
usek contains:
Fri Jun asdf asdf asdf asdfasdf
Fri Jun asdf asdf asdf asdfasdf
Fri Jul asdf asdf asdf asdfasdf
Fri Jul asdf asdf asdf asdfasdf
Fri Aug asdf asdf asdf asdfasdf
Fri Aug asdf asdf asdf asdfasdf
after the run, archived_data_log contains:
Fri Jun asdf asdf asdf asdfasdf
Fri Jun asdf asdf asdf asdfasdf
after the run, new_data_log contains:
Fri Jul asdf asdf asdf asdfasdf
Fri Jul asdf asdf asdf asdfasdf
Fri Aug asdf asdf asdf asdfasdf
Fri Aug asdf asdf asdf asdfasdf