Behind the scenes at Labs
Showing results for 
Search instead for 
Did you mean: 

HP Labs collaborates with HP Autonomy and HP Vertica to further develop integrated data management


Contributed by Simon Firth, freelance technology journalist


HP Autonomy logo.jpgA new addition to HP Autonomy’s IDOL (Intelligent Data Operating Layer) search and analytics platform, based on research conducted at HP Labs, is helping realize a vision in data management and analytics.


Today’s information-driven organizations want to shift big data analytics from a function performed only by domain experts to one they can apply to an expanding set of everyday, operational applications. But that requires a flexible, secure, and high-speed data engine that can economically acquire, combine, store and simultaneously analyze structured data, machine data, and new forms of unstructured, human information.


HP Vertica_logo.jpgRecently released as part of the HP IDOL platform, the HP IDOL UDx pack for HP Vertica lets users of the HP Vertica database management platform analyze unstructured data alongside transactional and semi-structured machine data, offering them a 360-degree view of the information they collect. 


meichun_hsu.jpg“Databases have long supported the standard SQL query language, and a community of developers as well as a stack of applications have been built around it,” explains Meichun Hsu, director of research in HP’s Analytics Lab. “But SQL was designed to let you query structured data. We’re exploiting SQL’s extensibility capability so that you can access IDOL functions to work on unstructured data while still within the Vertica SQL environment.”


A manager looking to retrieve customer comments about a product, for example, could now retrieve the full comment list from a database and also request a summary of them.  Or a sales executive could take a set of sales figures and, within Vertica, correlate them to records of news on specific topics held elsewhere.


The IDOL UDx Pack for Vertica forms one of the developer interfaces to HP’s HAVEn big data platform, which integrates HP’s IDOL, ArcSight, and Vertica products with Apache Hadoop.


malu'10.jpg“We’d already done a lot of research on multi-engine integration  as well as how to extend database functionality, and the HAVEn platform initiative was looking to demonstrate how IDOL’s capabilities could be accessed from SQL,” recalls Malu Castellanos, HP  senior researcher. “So we built a prototype that illustrated the concept.”  


“This approach makes the power of IDOL text and image analysis accessible from Vertica queries,” explains Girish Mundada, CTO of HAVEn. “It enables SQL-based applications to benefit from the broad range of analytics capabilities of HAVEn platform. ”


The result was a collaboration between the HP Labs team and engineers at both HP Autonomy and Vertica that led to the technology transfer that underpins the IDOL UDx pack for Vertica.


“One of our big research interests is in creating an integrated, open, scalable, and lower-cost big data platform,” notes Hsu. “This addition to HP HAVEn is pointing to one way that we might build it.”


“The HP Labs prototype was one of the first HAVEn multi-engine projects,” adds HAVEn CTO Mundada.   “We expect more innovations in multi-engine integration to be made by both HP Labs and the various product teams to deliver on our vision of HAVEn Platform.”


An early-access version of the HP IDOL UDx pack for HP Vertica is already available on the Vertica marketplace.



0 Kudos
About the Author


Online Expert Days - 2020
Visit this forum and get the schedules for online Expert Days where you can talk to HPE product experts, R&D and support team members and get answers...
Read more
HPE at 2020 Technology Events
Learn about the technology events where Hewlett Packard Enterprise will have a presence in 2020
Read more
View all