Behind the scenes @ Labs
Showing results for 
Search instead for 
Do you mean 

HP Labs collaborates with HP Autonomy and HP Vertica to further develop integrated data management

‎02-28-2014 03:59 PM - edited ‎09-30-2015 07:01 AM

Contributed by Simon Firth, freelance technology journalist

 

HP Autonomy logo.jpgA new addition to HP Autonomy’s IDOL (Intelligent Data Operating Layer) search and analytics platform, based on research conducted at HP Labs, is helping realize a vision in data management and analytics.

 

Today’s information-driven organizations want to shift big data analytics from a function performed only by domain experts to one they can apply to an expanding set of everyday, operational applications. But that requires a flexible, secure, and high-speed data engine that can economically acquire, combine, store and simultaneously analyze structured data, machine data, and new forms of unstructured, human information.

 

HP Vertica_logo.jpgRecently released as part of the HP IDOL platform, the HP IDOL UDx pack for HP Vertica lets users of the HP Vertica database management platform analyze unstructured data alongside transactional and semi-structured machine data, offering them a 360-degree view of the information they collect. 

 

meichun_hsu.jpg“Databases have long supported the standard SQL query language, and a community of developers as well as a stack of applications have been built around it,” explains Meichun Hsu, director of research in HP’s Analytics Lab. “But SQL was designed to let you query structured data. We’re exploiting SQL’s extensibility capability so that you can access IDOL functions to work on unstructured data while still within the Vertica SQL environment.”

 

A manager looking to retrieve customer comments about a product, for example, could now retrieve the full comment list from a database and also request a summary of them.  Or a sales executive could take a set of sales figures and, within Vertica, correlate them to records of news on specific topics held elsewhere.

 

The IDOL UDx Pack for Vertica forms one of the developer interfaces to HP’s HAVEn big data platform, which integrates HP’s IDOL, ArcSight, and Vertica products with Apache Hadoop.

 

malu'10.jpg“We’d already done a lot of research on multi-engine integration  as well as how to extend database functionality, and the HAVEn platform initiative was looking to demonstrate how IDOL’s capabilities could be accessed from SQL,” recalls Malu Castellanos, HP  senior researcher. “So we built a prototype that illustrated the concept.”  

 

“This approach makes the power of IDOL text and image analysis accessible from Vertica queries,” explains Girish Mundada, CTO of HAVEn. “It enables SQL-based applications to benefit from the broad range of analytics capabilities of HAVEn platform. ”

 

The result was a collaboration between the HP Labs team and engineers at both HP Autonomy and Vertica that led to the technology transfer that underpins the IDOL UDx pack for Vertica.

 

“One of our big research interests is in creating an integrated, open, scalable, and lower-cost big data platform,” notes Hsu. “This addition to HP HAVEn is pointing to one way that we might build it.”

 

“The HP Labs prototype was one of the first HAVEn multi-engine projects,” adds HAVEn CTO Mundada.   “We expect more innovations in multi-engine integration to be made by both HP Labs and the various product teams to deliver on our vision of HAVEn Platform.”

 

An early-access version of the HP IDOL UDx pack for HP Vertica is already available on the Vertica marketplace.

 

HP_Labs_insignia_developed-with_blue.jpg

0 Kudos
About the Author

Guest_Blogger

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Events
Aug 29 - Sep 1
Boston, MA
HPE Big Data Conference 2016
Attend HPE’s Big Data Conference to learn from peers in every industry and hear from Big Data experts and thought leaders in an exciting, energy fille...
Read more
Sep 13-16
National Harbor, MD
HPE Protect 2016
Protect 2016 is our annual conference and is the place to meet the world’s top information security talent, discuss new products and share information...
Read more
View all