Digital Transformation
Showing results for 
Search instead for 
Did you mean: 

Using HP's big data solution to fix problems with complex systems


Us humans are creating ever more complex systems. Which is great. But whenever these systems go wrong, it’s becoming harder to figure out where the problem lies. Big data can help because typically, these complex systems create a lot of data that we need to analyse quickly in order to diagnose problems.
Let’s look at how big data is being used to fix complex problems. 
HP Operations Analytics is an HP IT management product that uses both HP ArcSight and HP Vertica. HP ArcSight is used for the collection of applications and systems’ log files and HP Vertica for the analysis of all the log file, event and performance information. 


Because of increased use of cloud, BYOD mobile devices and in the future, machine to machine data traffic, we are faced with managing applications comprised of components over which we have little control (we can’t put monitors on cloud providers’ systems, or on smart devices we don’t own).


IT ops analytics diagram.png


We thus have to “record everything”. When there is a problem, HP Operations Analytics allows us to go back in time from the point of problem until we find the point when the issue started to build up. HP Operations Analytics uses the service hierarchy to filter the data that it looks at. For example, if an online check-in app is having problems, HP Operations Analytics will just look at data associated with dependent services. 

That’s IT Operations Analytics. HP big data solution is also used by telcos for their operations analytics too – operations analytics on their 3g and 4g networks...


KDDI Mobile (second largest mobile operator in Japan) records billions of records of information about its mobile data networks. It uses this to solve performance and “dead area” problems that its customers are having. Without the use of HP Vertica, KDDI simply wasn’t able to solve the problems fast enough for customers (it’s no good telling someone why they were having problems two hours ago – they are probably no longer in the same place, and so dont now care !!). Like with HP Operations Analytics, KDDI is recording “micro-transactions” and using these to solve problems.


Vodafone Ireland has started using HP big data in the same way too.
HP Service AnywhereHP’s service desk-as-a-service product uses both HP Vertica and HP Autonomy. Let’s look at the numerous ways in which HP Service Anywhere uses the Autonomy IDOL engine:
HP Service Anywhere diag.png
1. Fuzzy plain text searching in the end user self-service portal:  You type in plain text search and get back issues, collaborations and knowledge articles. This uses the Autonomy IDOL engine so that the search terms don’t have to exactly match what is in the knowledge store. 

2. Actionable “hot topics” : This is also powered by the Autonomy IDOL engine's analytics of unstructured data. IDOL is asked to look for clusters of topics. This cluster analysis is used in two ways within HP Service Anywhere:

- To find hot social Q&A topics. This clustering information regarding social Q&A topics is then used to create knowledge-base content. This is sometimes called “ticketless IT support” - IT support doesn’t wait for people to raise tickets. Instead, they monitor social sentiment and proactively fix problems and generate content for the end-user knowledge base. HP IT uses such an approach. It monitors the internal dialog within HP regarding the services it offers and looks for clustering, using the IDOL engine. Then, again using the IDOL engine, it does sentiment analysis on the clusters. Hot negative sentiment is then flagged up and HP IT proactively works on this.

- To find hot topics in the incident database. This is then used to proactively create problem records

3. CTI integration. Based on the phone number calling into the service desk, caller’s data and the requests they opened recently are shown. But more interestingly, as the conversation takes place, the service desk agent can type in comments and then articles, open issues, and known problems are shown to help resolve the issue on the phone. This is similar to the HP ES big data product that monitors your Microsoft Lync conversations, passes the text to the IDOL engine, and uses the meaning that comes back from IDOL to fetch related documents and corporate information. 
HP’s Gen8 servers : HP support takes telemetric data from customers’ Gen8 servers, if the customer allows them to do so. When a customer calls with a problem, the system looks for a pattern of events prior to the problem occurring. In big data analysis speak, the subject matter expert (the support person) is teaching the predictive system what to look for. Then, when the system sees the same combination of two or three events occurring again, at any customer site, proactive action can be taken to avoid problems.  (see diagram below [note : this diagram is not using real customer data])
gen8 scatter.png
There is a large amount of telemetric data involved, and so HP Support uses the HP Vertica product to analyse this data. 
In the next post, I look at how customers are using HP HAVEn to optimize complex systems. (My previous post looked at the use of big data for fraud and compliance management)

Mike Shaw
Director Strategic Marketing

linkedin.gifMike Shaw

0 Kudos
About the Author


Mike has been with HPE for 30 years. Half of that time was in research and development, mainly as an architect. The other 15 years has been spent in product management, product marketing, and now, strategic marketing. .

Jan 30-31, 2018
Expert Days - 2018
Visit this forum and get the schedules for online HPE Expert Days where you can talk to HPE product experts, R&D and support team members and get answ...
Read more
See posts for dates
HPE Webinars - 2018
Find out about this year's live broadcasts and on-demand webinars.
Read more
View all