Protect Your Assets
Showing results for 
Search instead for 
Do you mean 

Big Data Security Analytics Part 4: Visualization is Key

‎05-15-2014 09:30 AM - edited ‎06-09-2015 11:28 AM

Humans are visual hunters. Sight is our dominant sense and our brain structure is optimized for pattern recognition, these facts make data visualization one of the more powerful techniques we can apply to the big data security problem. There is also a factor of time scale, in traditional operational security monitoring you are dealing with a time period of 5-30 minutes; however, as the advanced attacker is aware of this limitation, you cannot stop "hunting" them down in your historical data, otherwise known as analytical time. 


Another power of data visualization is the ability to root cause analyze and remove. If you take millions of events that cover a period of months you are guaranteed that the vast majority of it will be legitimate activity. You can explore this data using multiple visual techniques and as you root cause analyze it as legitimate activity you can remove it from these visualization and slowly peel the onion back to the subset of data for which you cannot find an obvious root cause and this resulting data set will be of real interest for security events. 


There is an interesting phenomenon when you look at enterprise log data with visualization tool; computers have a hard time with random patterns but humans almost can't help but have bits of randomness in their activity. This is very visually recognizable in many cases and can be used to identify malicious activity. This is what makes simple visual exploration a powerful tool for identifying malicious activity in addition to algorithmic data analysis. The best approach is to conduct open ended data exploration and as you identify interesting items these can then be automated for consistent identification. 


Ultimately, the most powerful aspect of data visualization applied to big security data is that a data scientist is not required. A security subject matter expert has the best chance to find bad guys.


To explore the types of questions you can ask of your big data read the blog post: Important Questions for Big Security Data


Click here to learn more about HP HAVEn.


Thank you Chris Calvert for contributing this content.


Check out part 5 of this series: Big Data Security Analytics Part 5: The Challenges of Successful BDSA

0 Kudos
About the Author


Nov 29 - Dec 1
Discover 2016 London
Learn how to thrive in a world of digital transformation at our biggest event of the year, Discover 2016 London, November 29 - December 1.
Read more
Each Month in 2016
Software Expert Days - 2016
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
View all