HPE Alletra MP X10000 - high-performance object storage for data analytics and protection

[PiotrDrag]

In early 2023, HPE introduced a modular data management solution under the HPE Alletra Storage MP brand. Initially in block form, followed by file form a year later. Now it is time for data stored in object form – HPE Alletra Storage MP X10000. This solution, like its block and file counterparts, is supported by the standardized and modular hardware platform, HPE Alletra Storage MP, built from the ground up.

HPE Alletra MP X10000: a modern approach to data analytics and protection

The new HPE Alletra Storage MP X10000 object storage has been designed with large unstructured data sets in mind and optimized to enable enterprises to store and process object data faster. The exceptional performance of the X10000 is ideal for data lakes and modern data protection, where high processing performance and short data access times are crucial.

With this new offering, HPE further simplifies storage management based on a unified modular architecture for block, file, and object storage, along with intuitive management on the HPE GreenLake cloud platform.

Moreover, HPE is collaborating with NVIDIA to provide direct memory access (DMA) functionality between GPU memory, system memory, and the X10000 via the S3 object protocol, which is critical for AI-based applications. This direct access path will further increase system throughput, reduce latency, and enhance GPU utilization. This will be a unique solution on the market, as access is currently provided only through file protocols.

Flexibility and scalability

One of the key advantages of HPE Alletra Storage MP X10000 is its flexibility and scalability. By employing an innovative approach to distributed file&object arrays based on a disaggregated or modular architecture, users can easily adjust storage capacity according to changing business needs. This flexibility helps avoid excessive infrastructure investments while ensuring the ability to quickly respond to changes in data storage requirements. Furthermore, this approach provides a level of performance unprecedented in previous systems. This is crucial for all companies and institutions that handle large volumes of data, require fast access to it, and need dynamic allocation of computing power.

Another distinguishing feature is the use of a key-value database for data storage. On the one hand, this accelerates the operations of writing and reading objects. On the other hand, it allows for the addition of a wide range of metadata to each object without negatively impacting performance or data consistency, even with massive data sets that scale to petabytes.

 System atchitecture

Data security

A key element of data protection on the HPE Alletra Storage MP X10000 array is RAID Triple Parity, which allows for triple parity data protection within a single array. The use of this data protection technology significantly enhances capacity efficiency (usable capacity constitutes ~75% of the installed physical disk capacity). It also ensures a high level of data security and access. Calculating three parity data for each RAID group (in practice, a RAID group equals a disk shelf) provides the ability to lose up to three disks in each shelf composed of 24 disks, without losing the entire data set stored on them.

HPE Alletra Storage MP – hardware foundation of HPE Alletra Storage MP X10000

HPE Alletra Storage MP is built on a disaggregated Software Defined Storage architecture. The term "disaggregated" in this context means the ability to independently scale compute and storage capacities, leading to more efficient resource utilization and eliminating the need for significant redundancy. You simply buy what you need now and expand as necessary in the future. This hardware platform can therefore be used for various applications. In this article, we focus on its object storage incarnation, the HPE Alletra Storage MP X10000.

The key hardware features of the HPE Alletra Storage MP X10000 are:

• 2U chassis with 24 slots for NVMe SSD drives
• Two types of nodes:

- Compute node with either dual or single controllers

- Storage node/disk shelf

• IO module/controller (IOM) based on AMD processors: 

- Each node has two modules

- Single socket multi-core CPU - 64-core in compute nodes, 8-core in storage nodes

- Different RAM configurations - 512GB RAM in compute nodes, 64GB RAM in storage nodes

- 2 OCP adapters per module with two 100Gb ports per adapter (up to 8 ports per node)

• Universal drive carrier for all future SSD formats (U.2, E3.S 1T, E3.L 1T, etc.)
• 24 NVMe TLC drives (1.92TB, 3.84TB, 7.68TB, 15.36TB, 30.72TB) per disk shelf
• Two 100Gb switches (32 ports) to support the distributed array
• TPM (Trusted Platform Module) unit
• Chassis detection module through HPE GreenLake Cloud Platform 

The minimum configuration consists of two compute nodes equipped with 3 controllers, one storage node, and two switches. In terms of capacity, we are talking about a minimum of 100TB of usable capacity in the starting configuration.

Management

Finally, perhaps the most important architectural assumption of the system is that the management of every storage system should be done from a single console, the HPE DSCC (Data Services Cloud Console), deployed on the HPE GreenLake platform. This will provide clients with unified user experience regardless of the type of solution or scale of operations. Moreover, the console is so simple and easy to manage that it practically eliminates the need for investing in specialized technical knowledge to operate all types of storage. From the perspective of the entire company or organization, we gain global visibility of the entire data management environment.

That's how it looks from the perspective of an IT administrator.

The administrator first logs into the HPE DSCC console, where they select the appropriate application for managing the chosen type of storage (e.g., block, file, or object). All tasks can be performed within the selected application, which is fully maintained and updated by HPE. The hardware is essentially hidden under the layer of the HPE DSCC console. The administrator's primary concerns are ensuring the appropriate level of performance for their applications and the security of the entire environment, including access management.

Software

The brain of the HPE Alletra Storage MP X10000 system is software built from the ground up, based on a key-value store model. In this model, the key-value store implements the storage of object fragments (called chunks) and their metadata independently of the data transfer protocol, forming the core data layer. It is optimized for flash memory access, reducing write amplification thanks to a log-structured and extent-based approach.

Within the key-value store, there are namespace layers specific to individual protocols, such as S3/Object. These protocol layers are optimized for the semantics of each specific protocol, treating each one as a first-class citizen. This allows the HPE Alletra Storage MP X10000 system to leverage the advantages of each protocol without inheriting the drawbacks of another or stacking protocols on top of each other (such as running an object protocol on top of a file protocol or vice versa).

From a software development perspective, the entire system is built using containers. This is beneficial in terms of flexibility and scalability and opens many possibilities for utilizing the system's computational power in the future. With high-performance compute nodes, it will be possible to deploy data analysis applications directly on them (e.g., Apache Iceberg, Kafka, etc.).

HPE Alletra Storage MP X10000 is designed to provide balanced read and write performance, both for high throughput and small transactional operations. This means that large clusters are not required for workloads with a high number of write operations. This results in optimized performance regardless of the workload, allowing performance goals to be met without waste. One of the design goals of the HPE Alletra Storage MP X10000 array was to ensure high initial performance even with a relatively small starting environment (i.e., 3 compute modules and one disk shelf).

System architecture of HPE Alletra Storage MP X10000 

Use cases

Workloads associated with unstructured data are extremely diverse. Even within a specific category of applications, such as artificial intelligence, the nature of workloads can vary significantly. While many object storage systems prioritize throughput-oriented performance, HPE Alletra Storage MP X10000 provides high IOPS performance for small objects, as well as high throughput for larger objects and low latency for GET and PUT operations (< 2 ms).

In object storage, S3 buckets are a useful structure. However, in the case of certain object solutions, it is often necessary to use multiple S3 buckets to achieve maximum system performance. Typical unstructured workloads, such as analytics and data protection, often require a single bucket or a small number of buckets per application-specific unit, such as a single data store or backup chain.

For HPE Alletra Storage MP X10000, this is not a problem. Even a single bucket is sufficient to achieve maximum system performance. This eliminates the complexity associated with additional configuration to enhance system performance. This is particularly noticeable in write (PUT) operations. Compared to selected competing solutions, we can be up to 60 times faster in PUT operations for small objects using a single bucket.

The ability of HPE Alletra Storage MP X10000 to linearly scale a single bucket means that individual applications benefit from the same scalability as many applications or users.

Given the above, HPE Alletra Storage MP X10000 is ideally suited to handle a variety of applications that require high performance and low data access times—ranging from traditional data analytics applications to modern cloud and container environments. This system is also excellent for scenarios involving rapid recovery of large data sets after a failure or loss.

Data analytics

HPE Alletra Storage MP X10000 provides faster access to object data on a massive scale. With its high-performance all-flash object storage array utilizing a disaggregated, shared everything storage architecture, users of data analytics applications can leverage a single hardware platform to store data that can be shared across various applications (e.g., Apache Kafka, Spark, Iceberg, etc.). Moreover, the high-performance compute nodes and key-value store architecture enable the automatic addition of metadata to objects as required by specific applications or users.

Fast restore

HPE Alletra Storage MP X10000 provides immense recovery throughput, enabling faster recovery of even large amounts of lost data to quickly restore critical applications. Its S3-compatible object storage interface and deduplication based on the HPE Catalyst protocol significantly reduce the amount of stored data and streamline integration with any backup solution. HPE Alletra Storage MP X10000 is certified for backup by industry-leading providers, including Commvault and Veeam®, and is being verified with additional vendors.

Summary

HPE Alletra Storage MP X10000 is an advanced data management solution designed for modern enterprises and organizations that require flexibility, performance, and scalability in data storage. This system offers a unified platform that supports various workloads, from traditional business applications to modern cloud and container environments. With advanced features such as automated management, performance optimization, and cloud integration, HPE Alletra Storage MP X10000 enables organizations to manage data efficiently on a large scale while minimizing complexity and operational costs.

HPE maintains a steady course aimed at simplifying data management. The introduced storage solution, based on the modular HPE Alletra Storage MP hardware platform, has several unique features that provide users with undeniable benefits:

• Standardization on a single HPE Alletra Storage MP hardware platform for all types of storage (block, file, object) -> simplifies planning, expansion, servicing, increases availability, and accelerates delivery
• Disaggregated, software-defined architecture -> allows for independent expansion of compute and storage capacity, higher resource utilization, no need for significant redundancy (purchase as much as needed now and expand as necessary in the future)
• Management of every storage system from a single console on the HPE GreenLake Cloud Platform -> unified user experience, regardless of the type and scale of the solution, no need to invest in specialized technical knowledge for storage management, global visibility of the entire data management environment, and the ability to purchase additional data management and protection services

For those interested in HPE Alletra Storage MP X10000, I invite you to watch the introductory video on this technology.