HPE Ezmeral: Uncut
Matt_Hausmann

HPE Ezmeral: What a year it’s been!

We've come a long way in the past year. From its introduction to today, HPE Ezmeral has evolved into a unified, edge-to-cloud solution, delivering self-service analytics that can be consumed as a service via the cloud.

HPE Ezmeral Year in Review.png

In the modern business world, every company needs a data management infrastructure that can execute the necessary tasks to make data useful and actionable. What that infrastructure looks like will be individualized for each company; and therefore, a solution cannot be bought as an off-the-shelf turnkey product. Yet, what that infrastructure needs to deliver is well understood, which means no company needs to build it from scratch.

When we launched HPE Ezmeral in June 2020, we had a vision for productizing a solution with the key components that can “industrialize data science”. In other words, we wanted to make data analytics a repeatable and reproducible process, while allowing those components to be assembled, configured, and customized to support the needs of any business. 

But the more we’ve worked with our customers, the more we’ve come to believe we shouldn’t think of HPE Ezmeral as a loose platform with unrelated parts that work together. Rather, it is a unified solution that can be put to work in every organization’s environment. As a result, HPE Ezmeral today is a unified, edge-to-cloud solution that delivers self-service analytics and can be consumed as a service via the cloud. This article takes stock of how far we’ve come in crafting that vision and what we’ve learned so far.

Product Accomplishments

With a history of being a leader in the data management space, HPE drew on their legacy to unify two separate products. HPE Ezmeral started with a foundation of BlueData to create an open-source Kubernetes runtime engine for application modernization, analytics, and machine learning operations (MLOps).  We then added HPE Ezmeral Data Fabric (formerly MapR) that sought to bring together data from edge to cloud and deliver it to workloads. Users could now develop, deploy, and monitor analytics all from a single place. With these integrated capabilities, our customers can absorb any kind of data from a distributed global footprint of applications, IoT sources, and other sources to bring that data into useful forms. Their workloads, especially those based on ML and AI, now run effectively in a governed manner. 

As we developed this foundation and put it in the hands of customers, it rapidly became clear how our platform could be extended but remain a unified product. We expanded the platform to fill adjacent white space, grouped in two categories: data and analytics foundation and enterprise delivery and infrastructure optimization.

Data and analytics foundation:

  • Data connectivity

HPE provides a variety of data access options – including native access within the HPE Ezmeral Container Platform across tenants, external data taps into your data lakes or object stores, and from edge to cloud with the embedded HPE Ezmeral Data Fabric.

  • MLOps capability

We’ve enabled collaboration across multiple tenants to allow sharing of metadata behind the data models. HPE Ezmeral serves as the integration layer for this feature, allowing users to see the data comprehensively, regardless of where the models are being run. This includes being able to bring in metadata into our shared store plus out-of-the-box integrations with Kubeflow and MLFlow.

  • Building data pipelines

We have built end-to-end analytics pipelines with AirFlow that allow for the creation of pipelines using multiple applications. AirFlow is a workflow orchestration platform that enables complicated jobs running in various applications to be united coherently. 

  • Spark Optimizations

HPE expanded the capabilities and support for Apache Spark. By integrating the open-source Apache Spark 3.x operator into our container platform, HPE is the first and only company to deliver enterprise, on-premises Apache Spark on K8s. This capability allows our clients to immediately take advantage of the latest and greatest Spark features such as optimizations for GPU acceleration and native data access from S3.

Enterprise delivery and infrastructure optimization:

  • Tighter HPE GreenLake integration  

To give our clients more flexibility in how they deploy and consume analytics, we released three HPE GreenLake analytics offerings powered by HPE Ezmeral: Cloud Services for app modernization, ML Ops for ML/AI aaS, and Splunk for enhanced cybersecurity.  

  • Hybrid-native capabilities

We offer hybrid-native capabilities for applications, infrastructure, and data from edge to cloud. Specifically, we added the ability for multi-cloud deployments – allowing clients to import, manage, and monitor clusters across the public clouds.  We unified this with data fabric capabilities spanning edge to cloud with our global namespace. 

  • Simplified app modernization

In addition to providing an open solution that leverages open source components, we’ve made some product improvements via contributions to open source via App Workbench and KubeDirector. These products make it easier to import and build custom applications with a guided workflow for Kubernetes application containerization, allowing you to create your own custom images or source from existing libraries in just a couple clicks. 

  • Continuous compliance with Git

We added centralized policy management that makes it easier for clients to adhere to corporate standard policies with the Open Policy Agent (OPA) rule engine; the policies can be applied across tenants and clusters.

  • Enterprise security

We added an integration with Falco to provide out-of-the-box runtime security. This capability delivers anti-virus-like runtime security by default to the platform, providing additional threat detection and alerting, which can be fed into the policy management rule engine to ensure continuous compliance. Additionally, we’ve added support for air-gapped deployments.  

  • GPU optimization

We’ve always enabled GPU sharing to improve utilization of your accelerated infrastructure, but this past year we expanded the capability to support heterogeneous GPU versions and native fractionalization with multi-instance GPU (MIG).

  • Data fabric as a standalone

We also now offer HPE Ezmeral Data Fabric as a standalone solution, allowing customers to pair the scale-out data fabric with any compute on any other platform.

  • Technical validations  

We understand the value of fully validated technology stacks, and we released four new reference architectures across our HPE Apollo, HPE Edgeline, and HPE Synergy compute offerings. 

Recent acquisitions

To further enhance the power of HPE Ezmeral, we’ve bolstered the platform through recent key acquisitions that help make a more unified platform to meet a wider range of customer needs. As stated earlier, this began by combining BlueData and MapR. But, it didn’t stop there.

The acquisition of these open-source offerings for Zero Trust and security will be integrated into our product later in the year. This integration means users can interweave security into their data, apps, and platforms seamlessly. 

With the acquisition of Ampool, we offer SQL on Hadoop with Presto, enabling customers to have an in-memory SQL search engine for their modern data analytics factory. There’s also a huge value add with hundreds of data connectors, federated query capabilities, and a data catalog metadata service. 

This acquisition adds value around MLOps by supporting (with tooling and integration) a clearly defined process for defining ML models, putting them into production, and optimizing them during operations. 

ISV marketplace

One of the key premises of the HPE Ezmeral platform is that we must be open and able to incorporate new and existing technologies. We’re not just content with what the platform offers now; we’re constantly seeking ways to expand its capabilities. The HPE Ezmeral Marketplace is a crucial part of this strategy, which allows us to integrate different technologies into the HPE Ezmeral universe.

HPE Ezmeral Marketplace.pngFigure 1. HPE Ezmeral Marketplace. Visit the site for the latest and complete list of ISV partners.

Through the ISV marketplace, ISVs can come and validate their software on the HPE Ezmeral software platform. It runs through our container platform, which serves as the foundation for our approach. On average, we’ve been adding around five new ISVs per quarter, with functionalities that include analytics, security, data quality, or anything else needed for the end-to-end data pipeline. Through the marketplace, we’re able to fill in the gaps needed by our customers, recognizing that no enterprise is using just one tool to meet all their needs. 

Obsessed with our customers

Yet perhaps our proudest accomplishment is the many affirmations we’ve received from our growing list of customers, who have entrusted key parts of their data analytics solutions to us. We’ve won CRN awards (2020, 2021) for the past two years, been recognized as 2021 Value Leader from EMA, and received Honorable Mention by Gartner for HPE Ezmeral ML Ops. More importantly, we have a list of extremely pleased customers who have already realized significant benefits from using HPE Ezmeral. These customers include: 

  • Mercedes Benz and DXC Technology - Data fabric-based automated and autonomous driving solutions
  • ORock Technologies - FedRAMP certified data science as a Service solution
  • Ericsson - Digital wallet program
  • Bidtellect - Modernizing a legacy Hadoop estate to a modern data fabric 
  • Nanya Technologies - Data science for DRAM design with GPU sharing 
  • Carestream - AI aaS via HPE Greenlake for improved medical care  
  • GM Financial – machine learning and data analytics in production

The benefits our customers have experienced include: multi-million dollar savings in infrastructure costs, massive increases in data generation on a daily basis, boosts in the number of orders online retailers can process, and the ability to handle thousands of more queries per hour than in the past.

Lessons learned and continued focus on delivering value

Despite all these successes, we truly believe the best is yet to come for HPE Ezmeral. But it’s worth stepping back at this point and reflect on some of the lessons we’ve learned since our launch. In 2019 we had the vision, in 2020 we entered the market with a product, and in 2021 we are accelerating implementations and learning how our customers are creating value with the platform.

One important finding was that while containers are crucial, creating value from data is king. We put a heavy emphasis on containers and Kubernetes with our BlueData infrastructure. But while that was important, we realized it isn’t the main differentiating feature of HPE Ezmeral. HPE Ezmeral’s strength is that we unified it with an embedded data fabric and MLOps. These capabilities give our customers a modern full-stack analytics solution — something that they had been desiring for quite some time. Our ISV marketplace also allows us to constantly add functionality to the platform through other vendors, which doesn’t box our customers into just one set of offerings.

By providing flexibility with a fully unified platform (rather than just a collection of capabilities), HPE Ezmeral has become the data management and analytics platform that can fit the needs of any business.

Matt Hausmann

Hewlett Packard Enterprise

twitter.com/HPE_Ezmeral
linkedin.com/showcase/hpe-ezmeral
hpe.com/HPE_Ezmeral

0 Kudos
About the Author

Matt_Hausmann

Over the past decades, Matt has had the privilege to collaborate with hundreds of companies and experts on ways to constantly improve how to turn data into insights. This continues to drive him as the ever-evolving analytics landscape enables organizations to continually make smarter, faster decisions.