Protect Your Assets
Showing results for 
Search instead for 
Do you mean 

Big Data Security Analytics Part 2: Security Analytics Results From a Combination of Tools

Kerry_Matre ‎05-01-2014 08:21 AM - edited ‎06-09-2015 11:29 AM

A combination of tools

Finding the needles and examining the haystack is not achieved by a single tool. The data to be analyzed is gathered via security and business tools (IDS, FW, email server, social media scraper, etc.). These logs and captured data can be stored in a universal log management system and pushed to a SIEM such as ArcSight for real-time correlation and identification of threats.  The data can flow in raw form and as correlated SIEM output into a data system, such as Hadoop or Vertica


These systems work in different ways to enable questions to be asked of the data.  “Is the event volume uncharacteristic for this time of the year?” “Was there increased security activity leading up to our new product announcement?”  “Is there any tie between our attrition patterns and different attack life cycle event volumes?” In combination with a content analysis system such as HP IDOL, you can ask “What is the likelihood that intellectual property is leaving my company?” or “Is there a negative sentiment about my company out in social media that raises my overall threat score?” Answers to these questions can be correlated with alerts from the SIEM to elevate the severity level of low priority events.  Additionally, these tools can be utilized by parts of the company outside of the security organization and they will have other pieces that fit into the puzzle.


Which build approach works best?

Many organizations looking to tackle the big-data problem find themselves asking "Do I find the questions I want to answer before I begin my project or do I build my big data capability and then determine what questions I want to ask?"

If you set out on a big data integration project with a small set of questions in mind, then the scope is well-defined and the success criteria are set. Funding will be easier to come by for this type of build-out. However, you may be limiting the full capabilities of a security analytics solution.


If you first install all of the plumbing for big data then it will be a big cost justification up front with no success criteria identified at the beginning. However, once the solution is in place, answering questions becomes very cheap. More importantly, the costs (in terms of dollars and impact to the business) of asking what-if questions drop dramatically. Simple questions and answers come very fast allowing more questions to be asked and more lessons to be learned. Instead of assuming all dimensions and facets of the answers are known of the data before the questions, a type of Socratic Method can be applied to your data exploration. The rate and methods of data generation have changed radically in the last several years. Why would we assume this rate of change to decrease?  Being able to quickly ask, and answer, all sorts of investigative questions becomes a huge competitive advantage.


The approach will be different depending on the needs of the organization, but the capabilities of the data analysis architecture should not be crippled.


Learn more about how HP HAVEn can help you.


Coming soon: Big Data Security Analytics Part 3: Data Science & Putting Structure to the Problem

0 Kudos
About the Author


27 Feb - 2 March 2017
Barcelona | Fira Gran Via
Mobile World Congress 2017
Hewlett Packard Enterprise at Mobile World Congress 2017, Barcelona | Fira Gran Via Location: Hall 3, Booth 3E11
Read more
Each Month in 2017
Software Expert Days - 2017
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
View all