HPE GreenLake for File Storage opens a new chapter for analytics and storage

[StorageExperts]

Going beyond Hadoop: With the increasing workload demands of AI, new storage requirements have evolved. Today, HPE GreenLake for File Storage is ready for this next chapter in modern data analytics environments. Learn why.

–By Denise Ochoa Mendoza, Storage Solutions Engineer, HPE

In the last decades, the world of analytics has undergone rapid change. Remember when Hadoop, the batch processing framework for big data, was the go-to solution? It offered the ability to enable advanced data analytics and support new business models. Since then, data has continued to grow astronomically, with the need to go beyond batch processing evolving as well. The demands of today’s modern data analytic environments go beyond batch-oriented workloads ­­– and on to more agile ad-hoc queries, interactive queries, and real-time analytics to deep learning use-cases.

Things continue to change at a rapid pace, changing even faster, particularly as age of AI has officially arrived at all business levels. This is truly a new chapter for analytics and storage.   

Surpassing Hadoop

The Hadoop era marked the introduction of big data processing capabilities to mainstream organizations. Hadoop introduced concepts like MapReduce for tightly coupled storage and compute. It also used HDFS, Hadoop's distributed file system, which was implemented at a time when drives were faster than network adapters. This is no longer the case. The coupling of storage and compute provided a scale out problem as data continued to grow.

Most data pipelines originated with an inclusion of these process steps – starting first with data sources, along with data extraction, data transformation, and data-loading phases like in-batch processing. Next came the data processing-to-data storage stage, followed by data visualization and reporting.

Data pipeline architecture started off consisting of hardcoded pipelines using Extract, Transform, Load (ETL) patterns. This post-processing of data jobs could take hours or days. The process has now been modified to the current ETL approach, where the transformation takes place at the query layer. This allows different applications to have access to the raw data, enabling them to do only the transformations that they need. It also supports more agile ad-hoc queries, interactive queries, and real-time analytics. That said, a different storage approach is now warranted.

Storage built to last with an intuitive cloud experience

In the post-Hadoop era, you need storage designed for modern data analytics environments that incorporate AI. With HPE GreenLake for File Storage, you gain enterprise-grade, scale-out file storage designed to supercharge data-intensive environments. HPE GreenLake for File Storage is powered by VAST Data software running on HPE Alletra Storage MP hardware. It adheres to the use of Disaggregated Shared Everything (DASE) Architecture that allows independent scaling of cluster storage and compute – and addressed the challenges of tightly coupled storage and compute capacity expansion. This means that storage is now decoupled from compute resources and can scale out to exabytes, thus future proofing storage for data growth. 

Unlike in the Hadoop era where drives were faster than network adapters, HPE GreenLake for File Storage has all-NVMe speed for fast, predictable performance and no frontend caching or data movement between media. This means you can supercharge your most data-intensive apps with a read performance that is 80x faster than legacy NAS* and a read throughput of 100s of GB/sec.* In addition, you get six 9x data availability with no rebuild times for controller failures.

Data volumes continue to grow and the demands for real-time analytics and interactive queries rise. Hadoop's initial focus on batch processing, while revolutionary for its time, fell short in meeting these evolving requirements. This shift has been driven by the need to efficiently process data in real-time, accommodate ad-hoc queries, and provide a more seamless experience for users.

With HPE GreenLake for File Storage, you get a highly performant, extremely scalable file system ready for the demands of real-time analytics and interactive queries – coupled with the simplicity of the cloud. It’s truly storage ready to take you to the next chapter of modern analytical environments.

Watch this video to learn more about HPE GreenLake for File Storage.

*HPE Storage Substantiation

---

Meet Storage Experts blogger Denise Ochoa Mendoza, Storage Solutions Engineer, HPE

Denise is on the worldwide storage solutions team at HPE. She is passionate about technology and her current focus on Big Data and analytics.

Storage Experts
Hewlett Packard Enterprise

twitter.com/HPE_Storage
linkedin.com/showcase/hpestorage/
hpe.com/storage