Find out how HPE GreenLake for File Storage achieves high sustainability and ROI with enhanced efficiency at AI scale.
– By David Yu, Senior Manager Product Marketing, HPE Storage
Sustainability is a priority for HPE and not just an afterthought, especially in this age of AI. Everyone should be concerned about sustainability for many valid reasons. We all need to be responsible inhabitants of the planet and good stewards of the environment. HPE wants to help organizations like yours strive to be
The AI boom brings many new possibilities but also high power consumption and CO2 emissions. A Forbes article states that researchers at the University of Massachusetts Amherst analyzed various natural language processing (NLP) models to estimate the energy cost required to train them. The researchers concluded that training a single large language model emits around 600,000 pounds of CO2, equivalent to the emissions of 125 round-trip flights between New York and Beijing.
As your organization unleashes the power of AI, you must also consider sustainability and efficiency as you choose and invest in you AI infrastructure. This only reduces costs but also mitigates the impact on the environment. As AI applications and workloads process massive amounts of file data, a key part of the infrastructure is file storage. According to IDC, global data is expected to more than double in size from 2022 to 2026.1 As your organization races to keep up, you start to prioritize capacity first. This leads to expansion in data center footprint, more application and data silos, and correspondingly larger carbon emissions even as organizations are trying to manage tight budgets. Here is where HPE can help with its flagship file storage product.
Enter HPE GreenLake for File Storage: designed for sustainability
HPE GreenLake for File Storage delivers enterprise performance, simplicity, and efficiency, all at AI scale. It offers performance that spans all the stages of AI to unlock more value from all your data with faster time to insights and discovery for competitive advantage. HPE GreenLake for File Storage provides an intuitive cloud experience that empowers data scientists with increased productivity and optimizes efficiency for increased ROI. It also ensures high sustainability with capacity density, data center footprint, power, and cooling improvements for reduced emissions.
Efficient
Data-intensive AI models have an insatiable appetite for power, storage capacity, and rack space. And legacy NAS solutions with shared-nothing architectures cannot efficiently scale out to keep pace with the capacity density, cost/TB, and power efficiency demands of AI workloads.
HPE GreenLake for File Storage has DASETM (Disaggregated Shared Everything) architecture and HPE Alletra Storage MP modular, resilient hardware for independent scaling of performance and capacity. There’s no waste as it allows for easy scaling with the uncoupled addition of compute and storage nodes as needed. You can consolidate file data to eliminate storage silos, overprovisioning, and performance imbalance between compute and storage resources.
With its most recent enhancements, HPE GreenLake for File Storage can slash your AI storage costs with 4x the capacity per RU density and as much as 50% reduction in power consumption while delivering as much 2x the system performance per rack unit.2 It also lowers your carbon emissions with industry-leading data reduction, nondisruptive upgrades, and an AI storage as-a-service consumption model.
Scalable
The architecture is designed to grow to exabyte capacities, which future-proofs your storage investment. HPE GreenLake for File Storage scales up to 720 PB of effective capacity (with 3:1 data reduction) for large enterprise-scale AI file data. It delivers linear performance scaling while keeping overhead flat. The result is more scaling of performance than most organizations can ever consume. This solution eliminates migration issues and is designed to give you an infinite data lifecycle.
Fast
Under the hood, HPE Alletra Storage MP compute nodes are connected to HPE Alletra Storage MP storage nodes over an NVMe fabric. The storage nodes contain ultra-efficient, all-NVMe storage for blazing-fast performance, and every compute node can access all storage nodes and the data and metadata they store.
You can also help maximize GPU utilization (and therefore GPU ROI) with enterprise performance at AI scale. Support for optimized GPU utilization with InfiniBand, NVIDIA® GPUDirect, and RDMA accelerates AI workloads by boosting performance for model training and model tuning with faster checkpointing. Front-end host InfiniBand connectivity to networks, including the NVIDIA Quantum-2–based InfiniBand network platform, provides flexibility.
HPE GreenLake for File Storage supercharges your data-intensive workloads with enterprise performance at AI scale along with its enhanced efficiency to deliver a competitive advantage with more value from your data, increased productivity, and higher ROI and sustainability.
Sustainability: Flash longevity
HPE GreenLake for File Storage has data placement foresight. Because the software sees all the data that comes in, and because placement is across a single system and not partitioned across different nodes, data and stripes are efficiently placed in the storage according to their expected lifecycle.
The data that you need is always accessible, and it is placed in a way that extends the longevity of the underlying infrastructure. It’s intelligent data placement that limits write amplification and frees up stripes very efficiently (in their entirety rather than in fragmented pieces).This not only reduces garbage collection but extends flash longevity for greater sustainability.
Sustainability: Efficient data protection
The new erasure code of HPE GreenLake for File Storage breaks the cost/resilience tradeoff. It’s designed to maximize capacity efficiency without sacrificing fast rebuild times with a Storage Class Memory buffer for very large stripes.
If a 146+4 RAID stripe is used, the traditional Reed-Solomon erasure code would need to read all 149 remaining drives to rebuild a failed drive. The new erasure code eliminates this as the additional parity drives act as a “force multiplier” in rebuild times and only a quarter of the drives have to be read to rebuild a stripe. The system can rebuild a failed SSD without reading the entire very large stripe that the erasure code uses. This results in very low overhead (4 divided by 150 = .0266666 ~ 2.7%).
Sustainability: Superior data reduction
HPE GreenLake for File Storage further increases ROI with highly efficient data reduction that uses the unique Similarity algorithm. Similarity is important when you’re looking for data reduction in file data. Whereas many all-flash block systems rely on compression and deduplication and work well in environments for virtual machines (where you have many copies of operating systems or multiple copies of the same databases for QA and testing), in the unstructured world of files, there is a lot of data that will be similar but not identical.
Compression is fine-grained but local: you are reducing redundancy over a small piece of data. It’s also very computationally intensive. Deduplication is global over a large amount of data, but very coarse. Data is broken up into chunks of the same block size, and the system looks for exact matches. Similarity looks for data that is similar but not identical on both a global and fine-grained basis.
Take the example of DNA data. A DNA strand will have information that is mostly the same, but there will be differences at various points of uniqueness. With an algorithm looking for identical matches, that would be a mismatch. However, Similarity will spot blocks of data that are mostly the same, compress them together, track the changes between them, and store only what is common. This gives you the best of both worlds from compression, which is fine-grained but local, and deduplication, which is global but coarse.
To illustrate Similarity’s superior data reduction, here is a before-and-after comparison of a data footprint using Similarity vs. compression and deduplication:
Before
After
Example savings from Similarity: Up to 20:1 reduction for enterprise backup files; 8:1 for quantitative trading market data; 4:1 for log stores; 3.5:1 for Splunk; 3:1 for training data, VFX and animation data, and backup target; 2.5:1 for seismic and weather data; 2.1 for life sciences data and multi-tenant HPC environments.3
With investment protection, SSD longevity, efficient data protection, and superior data reduction, HPE GreenLake for File Storage offers impressive efficiency, productivity, and high ROI from your capital investments for enhanced sustainability.
Higher productivity and efficiency from human and capital resources
When considering file storage AI infrastructure, you absolutely need to secure performance at AI scale for today’s data-intensive workloads. But you also need to account for two additional factors: First, how will your solution enhance the productivity and efficiency of your valuable human resources across the board, from data scientists and subject-matter experts to IT staff? For that, you need efficient file storage infrastructure and simplified management. And second, you need to ensure that your capital investments will give you a high ROI and sustainability.
Enhanced sustainability for the AI age
As you evaluate and decide on your file storage strategy and AI investments, it’s important to have a comprehensive view that accounts for the effective utilization of all your resources, human and infrastructure alike. HPE GreenLake for File Storage is designed to deliver enhanced productivity and efficiency on both fronts, increasing your contribution to improved sustainability in the AI age.
To learn more
Check out:
HPE GreenLake for File Storage blog series
HPE GreenLake for File Storage: NVIDIA DGX BasePOD certified and NVIDIA OVX validated
HPE GreenLake for File Storage architecture
Technical overview: Inside HPE GreenLake for File Storage
And watch the HPE GreenLake for File Storage technical demo
1 IDC Market Forecast May 2022 Enterprise Organizations Driving Most of the Data Growth US49018922
2 These improvements are based on comparisons to the previous shipping product featuring the two-node, 2U controller boxes and 550TB 2U storage shelves using 30.72TB NVMe SSDs.
Meet David Yu, Senior Manager Product Marketing, HPE Storage
Storage Experts
Hewlett Packard Enterprise
twitter.com/HPE_Storage
linkedin.com/showcase/hpestorage/
hpe.com/storage
StorageExperts
Our team of Hewlett Packard Enterprise storage experts helps you dive deep into relevant data storage and data protection topics.
