- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Re: analyzing ipfilter logs, need shell script adv...
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-20-2010 06:24 AM
тАО04-20-2010 06:24 AM
I'm trying to come up with a way to import ipfilter logs into Microsfot Access. One problem is the logs sometimes have an extra column. For example the first two lines have nine columns, and the third line has eight columns:
apr 20 joe 0.0.0.0 2x port 15000 len 20
apr 21 bob 0.0.0.1 3x port 15000 len 25
apr 21 dave 0.0.0.3 port 15000 len 28
The first two lines have a column "2x" and "3x" that don't show up in the third line. This makes it hard to import the file into an Access table. I don't need that column with the *x*, I'd like to remove it from every line that contains the extra column while keeping the rest of the line. A simple "grep -v x filename > output.txt will produce a file with all the lines not containing an "x", but I want those lines, I just want to remove the expression that contains the "x".
What utility can do this? Can sed, awk, or cut go through each line and remove just the expression with the "x" character in it? I need to remove the entire expression such as "2x", "13x", and so forth, while retaining the rest of the information in the line. The columns are separated by spaces. Maybe there's a way to tell cut "check the number of columns, if there are nine then remove the fifth one, otherwise do nothing"?
Also if anyone knows of a tool that can take ipfilter logs and analyze them and create a report please let me know. Thanks,
Brian
Campus Card Services
Indiana University
Solved! Go to Solution.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-20-2010 06:49 AM
тАО04-20-2010 06:49 AM
Solutionworth doing, I always say.
$ echo 'apr 21 dave 0.0.0.3 port 15000 len 28' | sed -e 's/ [0-9]*x / /'
apr 21 dave 0.0.0.3 port 15000 len 28
$ echo 'apr 20 joe 0.0.0.0 2x port 15000 len 20' | sed -e 's/ [0-9]*x / /'
apr 20 joe 0.0.0.0 port 15000 len 20
You could also pretend that you're writing
something like a real computer program, and
use "read" to (try to) suck in nine tokens:
read a b c d e f g h i
use "if" to see if the last one is empty:
if [ -z "$i" ] ; then
and then write out the stuff you'd like,
accordingly.
As usual, there may be more than one way to
solve a problem like this.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-20-2010 06:53 AM
тАО04-20-2010 06:53 AM
Re: analyzing ipfilter logs, need shell script advice
If appears that it's the fifth column you don't want when there are more then eight columns. Hence:
# awk '{if (NF>8) {$5=""};print}' file
...will snip out the extra data.
Regards!
...JRF...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-20-2010 07:25 AM
тАО04-20-2010 07:25 AM
Re: analyzing ipfilter logs, need shell script advice
But it leaves in that unsightly extra space.
> [...] sed -e 's/ [0-9]*x / /'
Note that a more complex "sed" command could
effectively determine the token count, and
act accordingly:
$ echo 'aa bb' | \
sed -e 's/^\([^ ][^ ]*\) *[^ ]* *\([^ ][^ ]*\)$/\1 \2/'
aa bb
$ echo 'aa bb cc' | \
sed -e 's/^\([^ ][^ ]*\) *[^ ]* *\([^ ][^ ]*\)$/\1 \2/'
aa cc
Some shortening is possible if your "sed"
accommodates fancy regular expressions, but
I'll admit that it can get ugly.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-20-2010 07:38 AM
тАО04-20-2010 07:38 AM
Re: analyzing ipfilter logs, need shell script advice
Will this - [0-9]*x - account for any expression that ends in "x" with any number of integers preceeding it? Sometimes it's 2x, or 121x, or even 1232x. Maybe I should leave out the [0-9] and let it operate on any expression that ends in "x"?
Also, when I'm ready to use a file as input could you please give the syntax for the read command, would it be
while read
done < input file
Thanks,
Brian
Campus Card Services
Indiana University
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-20-2010 08:08 AM
тАО04-20-2010 08:08 AM
Re: analyzing ipfilter logs, need shell script advice
man sed
man regex
That was the intention. But, by itself,
it'll attack (almost) any token on the line.
And, as shown, you can play with this stuff
interactively, so if you have doubts, then
run the experiment.
> Maybe I should [...]
_You_ get to decide what you really want to
do. There aren't any month name
abbreviations which end in "x", but I don't
know all your users' names. Many things are
possible. Looking for "
safer than "
> [...] syntax for the read command [...]
What was wrong with:
read a b c d e f g h i
?
man
Look for "read".
There are oodles of shell scripting primers
out there on this new-fangled Inter-Web
thing.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-20-2010 08:10 AM
тАО04-20-2010 08:10 AM
Re: analyzing ipfilter logs, need shell script advice
> Steven: But it leaves in that unsightly extra space.
But who cares given that the ultimate goal is to import the filtered log into Microsoft software?
If that truly bothers you, you could do:
# awk '{if (NF>8) {$5=""};print}' file|tr -s " "
Regards!
...JRF...
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-20-2010 08:17 AM
тАО04-20-2010 08:17 AM
Re: analyzing ipfilter logs, need shell script advice
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-20-2010 08:49 AM
тАО04-20-2010 08:49 AM
Re: analyzing ipfilter logs, need shell script advice
I said "unsightly", not "fatal".
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО04-20-2010 09:15 AM
тАО04-20-2010 09:15 AM
Re: analyzing ipfilter logs, need shell script advice
Brian
Campus Card Services
Indiana University