Behind the scenes at Labs
Showing results for 
Search instead for 
Did you mean: 

HP Labs and HP IT launch Trafodion, an open source transactional database, built on Hadoop


Contributed by Simon Firth, freelance technology journalist




Declaring it “by far and away the coolest project at Discover,” HP CTO and Director of HP Labs Martin Fink recently announced the public beta release of Trafodion, an open source SQL-on-HBase data management engine that enables real-time transactional processing  on a massive scale.


Speaking at Discover 2014 in Las Vegas, Fink predicted Trafodion will redefine data processing. “If you are building a new application that focuses on Big Data, the odds are you are also going to need transactions as part of that application,” he said. “We’re giving you the tool to have an enterprise-grade, at-scale transaction system.”  


Trafodion builds on the scalability, elasticity, and flexibility of Hadoop, the popular open source framework for rapidly ingesting, storing, and analyzing large data sets. Hadoop, however, is designed around time-delayed batch processing, where immediate interactive analytics are not required, and thus doesn’t support the transactional data workloads that many enterprises need to run their business. Traditional operational databases, which excel at handling transactions, meanwhile struggle to cope with really large data sets. Trafodion extends Hadoop to provide guaranteed transactional integrity, enabling new kinds of big data applications to run on Hadoop.


“This lays the foundation for users with traditional Online Transaction Processing (OLTP) and Operational Data Store (ODS) workloads to either build new applications or move their current ones onto Hadoop, placing all their data, from big to small, structured to unstructured under one roof,” explains HP Labs research manager and Trafodion  program manager, Hema Ramaswamy.


An ANSI SQL engine running on top of Apache HBase, Trafodion supports storing and accessing large data sets through standard JDBC and ODBC connectivity. “It’s a fully-fledged ANSI SQL engine with an SQL compiler, a very mature optimizer and a highly parallel executor,” Ramaswamy notes. “It offers users the whole stack in building a big data solution, from top-to-bottom (SQL access to data store) on Hadoop that is completely open source.”


“Trafodion” is the Welsh word for transaction – a name chosen to emphasize the differentiation Trafodion provides in closing a critical gap in the Hadoop ecosystem.


The project, incubated at HP Labs, is a partnership between HP Labs and HP IT. Also speaking at Discover, Robert Youngjohns, GM of HP Software, described the collaboration that got the product – which amounts to some four million lines of code and is the first of its kind – up and running in little over 6 months, leveraging 20+ years of database HP technology. “We think we’ve got something really interesting,” he said, “and we’re now releasing it as an open source project so we can get your feedback as we take it to the next level.”


Making it open source is key, Ramaswamy adds. “It lets HP collaborate with others in the HBase community to build a transaction management layer that is useful to everyone. Businesses can use it without getting locked into a proprietary software, which is very attractive to them, and it helps us, because if we build a community of users, people are much more likely to use the technology and advance it for things that we never dreamed of – which is really exciting.”


The Trafodion team is now actively encouraging the open source community to try out Trafodion for their new applications, offer feedback, and help enhance the technology by contributing code, tests, documentation, or bug fixes. 


While in public beta, Trafodion will remain within the HP Labs’ fold. Team members have plenty to keep them busy – they plan to build one or more proofs of concept for the technology and harden the software so that it’s production-ready. And they’ll continue to collaborate with the open source community.


“We met a lot of people at Discover who were really excited to have a full-blown, relational database available on Hadoop,” says Ramaswamy. “Customers got to see Trafodion running an HP-IT security application working alongside other HP and partner products on HP hardware – they’re ready for access to a system like this.”





Related Links:



Apache HBaseHP_Labs_insignia_developed-with_blue.jpg



0 Kudos
About the Author


Online Expert Days - 2020
Visit this forum and get the schedules for online Expert Days where you can talk to HPE product experts, R&D and support team members and get answers...
Read more
HPE at 2020 Technology Events
Learn about the technology events where Hewlett Packard Enterprise will have a presence in 2020
Read more
View all