Fueling AI innovation with HPE Data Fabric Software

[Ashwin_Shetty]

Get the latest news on updates to HPE Data Fabric Software and discover how to empower AI with faster data preparation and management for your AI workloads.

Being a hybrid enterprise has emerged as the new normal as businesses optimize on-prem and across cloud to the edge. As data and the number of sources and formats continue to increase, organizations are struggling with some key challenges – as an elevated level of complexity comes from visibility, accessing, securing, governing, and managing data across various sources, formats, and locations. This results in your IT teams spending most of their time with administrative tasks, such as discovering, copying, normalizing, compliance, and governance rather than providing an optimized data landscape that delivers data insights.

At the same time, artificial intelligence (AI) and generative AI (GenAI) are revolutionizing every industry by driving efficiency with streamlined processes. Modern AI and analytics applications thrive on a diverse data set, requiring a mix of structured, unstructured, real-time, and historical information.

However, fragmented data silos arise from the complexity of data distributed across hybrid environments, hindering insights and business growth. This intricate data landscape significantly increases the burden of building and managing an effective data infrastructure both architecturally and operationally. Data fragmentation resulting from disparate data silos hinders efficient data access, impedes data-driven decision-making. This can limit your organizations ability to unlock valuable insights and achieve your full potential.

Enter HPE Data Fabric Software – Accelerating data driven insights

What is HPE Data Fabric Software? It's an intelligent data plane delivering on the needs of the hybrid data lakehouse through radically simplified management, control, and governance for analytic and AI applications across all enterprise storage assets.

HPE Data Fabric Software allows you to conquer today’s hybrid challenges by replacing multiple point products with a single solution optimized for hybrid data analytics. Data Fabric helps users access, manage, organize, and govern enterprise data, across a variety of formats, in a single, consistent, easy to use, edge to cloud data plane optimized for AI and analytics workloads. Designed to be target agnostic, this solution can be deployed on-premises, in co-locations, and on Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform™ (GCP™), and HPE GreenLake, and across edge locations. This in turn allows your users to visualize everything and access it from anywhere.

What’s new with HPE Data Fabric software?

The latest version of HPE Data Fabric Software continues to build on its foundation as a trusted enterprise-grade solution to deliver data lake houses. We’ve been talking with you to learn how we can enhance the product to better meet your needs. Thank you for helping us with your feedback. We have made significant improvements to the offering based on your feedback.

I am excited to introduce the following features that are a part of today’s storage announcement at NVIDIA GTC:

1. Supporting PC-AI initiative through adoption of Apache Iceberg – The new release of Data Fabric Software Fabric is the data backbone of the HPE Private Cloud AI data Lakehouse and provides an iceberg interface for PC-AI users to data hosed throughout their enterprise. This unified data layer allows data scientists to connect to external stores and query that data as iceberg compliant data without moving the data. Apache Iceberg is the emerging format for AI and analytical workloads. With this new release Data Fabric becomes an Iceberg end point for AI engineering. This makes it simple for AI engineering data scientists to easily point to the data lakehouse data source and run a query directly against it. Data Fabric can access the parquet files (open-source file format) in any file or object store join them and make them appear as Iceberg compliant stores. Data Fabric takes care of metadata management, secure access, joining files or objects across any source on-premises or in the cloud in the global namespace.
2. New dashboards exposing metadata for governance and compliance – Data Fabric collects audit data from the global namespace and makes it accessible as iceberg-compliant parquet data. This allows any Iceberg compliant query, dashboarding or AI algorithm access to compliance data from the Data Fabric. In this release, we include a dashboard providing out-of-the-box insights to compliance data through the Data Fabric’s User Interface. This provides organizations with end-to-end visibility, security, and compliance monitoring – without additional tooling.
3. Integrating with HPE Alletra Storage MP X10000 – In this latest update, the HPE Data Fabric unified global namespace has been expanded to support HPE Alletra Storage MP X10000. This extends the GNS’s ability to manage data across diverse storage environments. With HPE Data Fabric, you can now take advantage of heterogeneous read-and write-access to object systems as well as the option to integrate the X10000 as a high-performance tier for advanced analytics projects.
4. Expanding the hybrid data lakehouse vision – It’s easy to deploy Data Fabric easily in the cloud or at the edge in hardware limited environments. Data Fabric is a 100% software-defined platform and is highly configurable. Users can deploy Data Fabric easily through the UI, installer or even have the flexibility to build custom scripts like Chef or Bash to deploy optimally in the cloud, on-prem and on edge devices. Organizations can quickly deploy AI-ready data Lakehouse anywhere without complexity.
5. Enhancing HPE l Ecosystem Pack – This is a collection of open-source tools and engines that integrate with the Data Fabric. The Ecosystem Pack enables provides a set of industry standard workloads that can run on Data Fabric nodes. In this release we have expanded set of open source tools like JWT authentication support for secure access, Apache Flink for real time streaming analytics, and pre-integrated data connectors to accelerate analytics. The Ecosystem Pack improves developer productivity by reducing time spent on configuring and integrating third-party tools.

What are the HPE Data Fabric Software differentiators?

Transparent and consistent access for AI and analytics workloads across hybrid data

• Unified data view and management – provides a unified view of data across multiple storage systems, simplifying data management and reducing complexity, making it simple to create and execute complex data pipelines reducing latency and cost
• Scalable infrastructure – enables users to seamlessly grow their data lake house without compromising performance or scalability.
• Software defined – provides a 100% software defined platform that integrates with existing storage infrastructure

Integrated and end to end solution for modernized AI and analytic workloads

• Data lakehouse designed for modern workloads – provides your users with real-time insights and decision-making capabilities by integrating and supporting data engineering and analytical tooling within the platform
• Multi-format support – supports multiple protocols in a single deployment including NFS, HDFS, S3, POSIX, HBase, or a document DB, allowing it to support many AI and analytics use cases in a single platform
• Comprehensive solution – eliminates the need for third-party integrations by providing a complete solution for enterprise-wide data management and analytics requirements

Secure governance, compliance, and data plane optimization

• Single point for compliance and governance – provides a single point for collecting, analyzing, and enforcing compliance and governance for data across any data stores represented in the Global Namespace
• Consolidated data and security – unifies data view and security across data landscape in a single model
• Robust security and compliance – meets stringent security and compliance requirements with built-in access controls and encryption capabilities

Taking a comprehensive data management approach 

With HPE Data Fabric Software, we’re enabling you to transform your approach in managing applications and infrastructure. You can orchestrate compute, provision storage resources, configure networking and protect workloads with unified view and management. This helps to break down all data silos with agility and cloud operational experience to help you to remove complexity and focus on your business priorities. 

Count on a comprehensive and unified data platform that empowers your organization to overcome siloed data and accelerate AI initiatives. By addressing the challenges of data fragmentation, helping ensure data quality while providing scalable and secure infrastructure, HPE Data Fabric Software enables your businesses to derive maximum value from your data.

Learn more

Read the Build a robust data foundation for AI excellence

Ashwin Shetty
Hewlett Packard Enterprise

twitter.com/HPE_Storage
linkedin.com/showcase/hpestorage/
hpe.com/storage