Behind the scenes @ Labs
Showing results for 
Search instead for 
Do you mean 

HP Labs collaborates with HP Autonomy and HP Vertica to further develop integrated data management

Guest_Blogger ‎02-28-2014 03:59 PM - edited ‎09-30-2015 07:01 AM

Contributed by Simon Firth, freelance technology journalist


HP Autonomy logo.jpgA new addition to HP Autonomy’s IDOL (Intelligent Data Operating Layer) search and analytics platform, based on research conducted at HP Labs, is helping realize a vision in data management and analytics.


Today’s information-driven organizations want to shift big data analytics from a function performed only by domain experts to one they can apply to an expanding set of everyday, operational applications. But that requires a flexible, secure, and high-speed data engine that can economically acquire, combine, store and simultaneously analyze structured data, machine data, and new forms of unstructured, human information.


HP Vertica_logo.jpgRecently released as part of the HP IDOL platform, the HP IDOL UDx pack for HP Vertica lets users of the HP Vertica database management platform analyze unstructured data alongside transactional and semi-structured machine data, offering them a 360-degree view of the information they collect. 


meichun_hsu.jpg“Databases have long supported the standard SQL query language, and a community of developers as well as a stack of applications have been built around it,” explains Meichun Hsu, director of research in HP’s Analytics Lab. “But SQL was designed to let you query structured data. We’re exploiting SQL’s extensibility capability so that you can access IDOL functions to work on unstructured data while still within the Vertica SQL environment.”


A manager looking to retrieve customer comments about a product, for example, could now retrieve the full comment list from a database and also request a summary of them.  Or a sales executive could take a set of sales figures and, within Vertica, correlate them to records of news on specific topics held elsewhere.


The IDOL UDx Pack for Vertica forms one of the developer interfaces to HP’s HAVEn big data platform, which integrates HP’s IDOL, ArcSight, and Vertica products with Apache Hadoop.


malu'10.jpg“We’d already done a lot of research on multi-engine integration  as well as how to extend database functionality, and the HAVEn platform initiative was looking to demonstrate how IDOL’s capabilities could be accessed from SQL,” recalls Malu Castellanos, HP  senior researcher. “So we built a prototype that illustrated the concept.”  


“This approach makes the power of IDOL text and image analysis accessible from Vertica queries,” explains Girish Mundada, CTO of HAVEn. “It enables SQL-based applications to benefit from the broad range of analytics capabilities of HAVEn platform. ”


The result was a collaboration between the HP Labs team and engineers at both HP Autonomy and Vertica that led to the technology transfer that underpins the IDOL UDx pack for Vertica.


“One of our big research interests is in creating an integrated, open, scalable, and lower-cost big data platform,” notes Hsu. “This addition to HP HAVEn is pointing to one way that we might build it.”


“The HP Labs prototype was one of the first HAVEn multi-engine projects,” adds HAVEn CTO Mundada.   “We expect more innovations in multi-engine integration to be made by both HP Labs and the various product teams to deliver on our vision of HAVEn Platform.”


An early-access version of the HP IDOL UDx pack for HP Vertica is already available on the Vertica marketplace.



About the Author


Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the Community Guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.