- Community Home
- >
- Servers and Operating Systems
- >
- Operating Systems
- >
- Operating System - HP-UX
- >
- Geo Lookup
Categories
Company
Local Language
Forums
Discussions
Forums
- Data Protection and Retention
- Entry Storage Systems
- Legacy
- Midrange and Enterprise Storage
- Storage Networking
- HPE Nimble Storage
Discussions
Discussions
Discussions
Forums
Forums
Discussions
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
- BladeSystem Infrastructure and Application Solutions
- Appliance Servers
- Alpha Servers
- BackOffice Products
- Internet Products
- HPE 9000 and HPE e3000 Servers
- Networking
- Netservers
- Secure OS Software for Linux
- Server Management (Insight Manager 7)
- Windows Server 2003
- Operating System - Tru64 Unix
- ProLiant Deployment and Provisioning
- Linux-Based Community / Regional
- Microsoft System Center Integration
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Discussion Boards
Community
Resources
Forums
Blogs
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-30-2009 02:12 PM
тАО09-30-2009 02:12 PM
Geo Lookup
This thread is a successor of -
http://forums13.itrc.hp.com/service/forums/questionanswer.do?threadId=1373850
I found this nice little utility called geoiplookup which maps the IP addresses to country/country codes.
After fraudulent attack from country X on our website, we thought of having a way to monitor user pattern to see if a user is consistently logging in from country A and suddenly shows up in country B, it should alert us and we should monitor for any fraudulent activity on that account.
For this I want a script that gets all the IP addresses for that day and associated account id from the web logs and updates a master file with that information.
This script should also have the ability to alert us (through mail) once it puts in an entry that is duplicate in terms of an account id but has a different country code associated with it.
Here is how our web logs look like -
where IP is the first field and 670612718 is the account id that I need to update into the Master file.
Can someone help in scripting this for me.It's easier for me to get the IP but I would like to get the account id which I am not able to get consistently given the nature of the logs.
And on how to fulfill the condition about alerting us.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-30-2009 02:21 PM
тАО09-30-2009 02:21 PM
Re: Geo Lookup
Col1 | Col2
Account ID | Country Code
The script gets a list of IP's(& account id) , does geoiplookup and gets the associated country code and updates the file, but alerts us if the account id already exists in the file but is for a diff country(code).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-30-2009 02:31 PM
тАО09-30-2009 02:31 PM
Re: Geo Lookup
webN.access_log.20090929.gz
And there are 8 web servers in total and want this to run on a central log server which has all the web access_logs from the previous day.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-30-2009 02:36 PM
тАО09-30-2009 02:36 PM
Re: Geo Lookup
So the master file should have -
IP | Account ID| Country Code
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-30-2009 03:59 PM
тАО09-30-2009 03:59 PM
Re: Geo Lookup
Thanks,
Allan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО09-30-2009 05:31 PM
тАО09-30-2009 05:31 PM
Re: Geo Lookup
If they aren't consistent, how are we going to figure this out? What's the pattern?
Would like to have the IP address in the master file ...
>So the master file should have:
>IP | Account ID | Country Code
You need to make sure of your requirements. Then you can decompose your scripting tasks.
Are you assuming you will start out with an initially empty "master" file?
>The web logs are gzipped
So, you need one part to read the logs and extract the IP and account ID.
Another part would be to look up in the master file.
Another part to do geoiplookup.
How big will this be? If you have 10s of thousands of records, you need some type of quick lookup, a database.
>And there are 8 web servers ... from the previous day.
I assume this isn't real time and you can do each log one at a time.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
тАО10-01-2009 10:36 AM
тАО10-01-2009 10:36 AM
Re: Geo Lookup
ANSWERS BELOW
>to get the account id which I am not able to get consistently given the nature of the logs.
If they aren't consistent, how are we going to figure this out? What's the pattern?
I meant that there are some entries which dont have the accountids in them, and I need to parse out the lines which have those ids.
The lines which have those ids, has a certain pattern as shown/explained below -
The account id will always be the 3rd entry from /(slash) - with t15 being server-number, v33.109- code version and 670612718 being the account id.
/t15/v33.109/670612718
Would like to have the IP address in the master file ...
>So the master file should have:
>IP | Account ID | Country Code
You need to make sure of your requirements. Then you can decompose your scripting tasks.
Are you assuming you will start out with an initially empty "master" file? YES
>The web logs are gzipped
So, you need one part to read the logs and extract the IP and account ID.
Another part to do geoiplookup.
Another part would be to look up in the master file. AND NOT POPULATE IF THE ENTIES ARE SIMILAR AND ALERT IF THE ACCOUNT ID EXISTS BUT IS FOR A DIFFERENT COUNTRY CODE SO THE GEO IP LOOKUP HAPPENS BEFORE THIS STEP.
How big will this be? If you have 10s of thousands of records, you need some type of quick lookup, a database.- We want to limit this to a TEXT file. If its easier for you to use the DB then I will let you handle that way.
>And there are 8 web servers ... from the previous day.
I assume this isn't real time and you can do each log one at a time. YES , this will happen only when the logs for that day have been downloaded on a central log server.