Traditional storage solutions are straining to keep pace with today’s demands. HPE Services can help you take your storage strategy to the next level with a wide variety of innovative approaches.
By Kyaw Thiha, Chief Solution Architect, Cloud Native Computing Practice Area, HPE Advisory & Professional Services
The need for massively scalable and flexible storage is a critical challenge for enterprises, since traditional storage platforms strain to meet such needs. Hewlett Packard Enterprise has been working closely with enterprises on this, and offers massively scalable modern storage solutions with scalability, performance, reliability, and security to address the challenges that enterprises face. HPE delivers a variety of storage offerings, such as the HPE Alletra series, as well as a wide range of partner offerings. I’ll describe some of these HPE storage solutions in this blog, and I’ll explain how HPE Services can help you realize the most value from them.
In addition, it’s clear that there is a growing need for open-source, software-based storage solutions with the use of storage hardware, and here again HPE Services can help. This article provides a high-level overview of one of the fastest-growing open-source software-based storage solutions: Ceph.
A comprehensive range of storage solutions
There are many approaches in the storage industry aimed at meeting businesses’ requirements, including the shift of devices to SSD, NVMe, cloud/hybrid cloud storage, and software-defined storage. HPE works closely with enterprises’ requirements and demands to address the challenges with a combination of offerings centred on data.
HPE offers a range of storage solutions to meet the diverse needs of organizations. Some of our leading storage solutions are as follows:
- HPE Primera Storage: a high-end, all-flash storage platform designed for mission-critical applications. It delivers extreme resiliency, high performance, and scalability to meet the demands of modern data-driven businesses. HPE Primera incorporates AI-powered intelligence for automated management and predictive support.
- HPE Nimble Storage: a hybrid flash platform that combines flash storage with predictive analytics to deliver high performance, high availability, and data efficiency. It leverages technologies like HPE InfoSight AI and Adaptive Flash to provide intelligent data management, automation, and easy scalability.
- HPE Alletra Storage: a modern storage platform that delivers high performance, scalability, and intelligence. It is designed to deliver high-performance, scalable, and intelligent storage for modern data-intensive workloads. It combines the power of NVMe and SCM (storage class memory) technologies with advanced data optimization and AI-driven operations to meet the demands of data-intensive workloads.
An extensive partner ecosystem
HPE completes the storage paradigm by not only providing industry-leading solutions but also establishing strong partnerships with various storage providers. HPE offers a comprehensive choice of storage options to customers.
- Veeam is a leading provider of backup, recovery, and data management solutions. The partnership allows customers to integrate Veeam's data protection and backup software with HPE's storage products, enabling efficient data protection and disaster recovery solutions.
- Commvault is a data management and backup software company that delivers integrated data protection solutions. Customers can use Commvault's software to manage and protect their data stored on HPE storage systems.
- Cohesity provides a data management and secondary storage platform, to provide data backup and management solutions. The partnership enables customers to simplify data management and offers comprehensive data protection solutions.
- Scality is a software-defined object storage company that provides solutions for managing and storing large amounts of unstructured data. HPE built a partnership with Scality to integrate Scality's RING, a soft-defined and distributed file and object storage platform, with HPE's hardware infrastructure. Thanks to the partnership, customers will benefit from a scalable and flexible storage solution to handle massive amounts of unstructured data.
These are just a few highlights of HPE's storage solutions. HPE offers a comprehensive range of storage offerings that cater to various requirements.
All the features you need
The solutions come with features like data protection, scalability, intelligent management, and integration with hybrid and multi-cloud environments to help organizations efficiently store, protect, and manage their data.
Here are some key features and benefits of the HPE storage solutions:
- Performance: HPE storage systems provide exceptional performance to handle demanding workloads by utilizing advanced technologies such as NVMe and SCM to deliver low latency and high throughput, enabling faster data access and processing.
- Scalability: HPE storage solutions are built to scale seamlessly as your storage requirements grow. They offer flexible configurations and can scale up not only capacity but also performance without disruption.
- Data optimization: HPE storage incorporates intelligent data optimization features to enhance efficiency with inline data deduplication and compression. These optimization techniques help lower costs and improve performance.
- High availability: HPE storage systems are designed to provide high availability and ensure business continuity. There is no single point of failure –from power supplies, to controllers, to storage/RAID, to data – with replication.
- Data management and analytics: HPE storage solutions come with comprehensive management and analytics capabilities which provide a unified view of the storage infrastructure, simplifying administration tasks and enabling efficient provisioning and monitoring. The analytics features offer insights into storage performance, capacity utilization, and data access patterns, empowering IT administrators to optimize storage resources and make informed decisions.
- Multi-cloud integration: HPE storage solutions are designed to seamlessly integrate with hybrid and multi-cloud environments. They support various data services and protocols, enabling easy data mobility between on-premises and cloud storage. This flexibility allows organizations to leverage the benefits of cloud storage while maintaining control over their data.
- AI-driven operations: HPE incorporates artificial intelligence capabilities through the integration of HPE InfoSight. This AI-driven platform analyzes data from the storage infrastructure to predict and prevent potential issues, optimize performance, and provide proactive support. It helps to reduce downtime, improve storage efficiency, and enhance overall system reliability.
With this modern and robust storage infrastructure, HPE offers storage solutions that fulfil the demanding requirements of data-intensive workloads. Whether it's for databases, analytics, virtualized environments, or AI/ML applications, HPE storage solutions offer the performance, scalability, and intelligence needed to handle critical business data effectively.
In addition to these storage offerings, HPE provides solutions that leverage open-source-based software-defined storage solutions such as Ceph. It is one of the fastest-growing open source software-based storage solutions.
Ceph storage solution
Ceph is designed to overcome the limitations and fulfil the requirements of modern storage solutions such as massive scalability, high IOPS/performance, data reliability, uptime and availability.
A Ceph Storage Cluster is built on the Reliable Autonomic Distributed Object Store (RADOS), a software layer that manages object storage across the cluster. Ceph uses the CRUSH algorithm, an intelligent data distribution mechanism, to manage data placement across the cluster and the hardware. By creating the CRUSH map of data to storage drives, bottlenecks and hotspots are avoided. Ceph has no issue handling billions of objects and beyond without performance penalty.
With the CRUSH algorithm, Ceph computes where the data should be written to or read from. Instead of storing metadata, CRUSH computes metadata on demand, thus removing the need for a centralized server/gateway or broker. It allows Ceph clients to compute metadata, also known as CRUSH lookup, and communicates with storage devices directly. It increases performance and responsiveness.
Ceph also comes with data replication (including erasure coding) for data reliability and availability, instead of traditional RAID.
Administrators can define a set of clear rules for how data is distributed throughout their clusters. Ceph provides a web-based management and monitoring portal where various aspects and resources can be inspected and administered. Block, object, and file storage solutions are provided from a single Ceph cluster.
Object storage
Ceph's object storage is compatible with popular interfaces, including Amazon S3 and Swift RESTful APIs. Workloads and applications can interact with object storage data over these interfaces.
Block storage
Ceph block devices are thin-provisioned. It is resizable and stored data striped over multiple physical disks and servers. Ceph block devices leverage RADOS capabilities including snapshotting, replication, and striping for maximized availability. Ceph block storage can be mounted on operating systems and virtual machines by using iSCSI via RADOS Block Device interface.
File system
It is a POSIX-compliant file system and comes with dynamic rebalancing and high performance. It can be used to share files, data and directories. See the figure below.
Ceph components
A Ceph storage cluster is constructed with the following components:
- Cluster monitors (MONs) maintain the map of the cluster state, keeping track of node status, data placement and authentication.
- Managers (MGRs) maintain cluster runtime information.
- RADOS Gateways (RADOS-GWs) provide object storage APIs.
- iSCSI gateways (iSCSI-GWs) provide iSCSI protocol to access Ceph's distributed block storage from client machines.
- Metadata servers (MDSs) store metadata for the Ceph File System, mapping filenames and directories of the file system to RADOS objects and enabling the use of POSIX semantics to access the files.
- Object storage devices (OSDs) store data in the Ceph cluster and handle data replication, erasure coding, recovery, and rebalancing.
Ceph is massively scalable
Ceph addresses the requirements of modern storage solutions by having a unique design. Traditional storage solutions usually come with a single entry point which receives the client requests. It can be a single point of failure and a bottleneck for performance and scalability.
Even though Ceph’s clients see the Ceph cluster as a single logical storage solution, they are directly communicating with:
- A metadata server to perform metadata operations
- Object storage devices to perform reads and writes
This improves overall scalability and performance. Ceph uses the CRUSH algorithm to eliminate centralization. Because of the CRUSH algorithm, clients and OSD daemons are not required to rely on centralized lookup tables. The object location can be computed efficiently and distributedly. The CRUSH algorithm enables massive scale by distributing the object and IO with better data management mechanisms.
Ceph is designed with a highly adaptive distributed metadata cluster architecture that improves the scalability of metadata access, and the scalability of the entire system.
Traditional storage scaling-up is not adequate for modern storage demands. Ceph can be scaled out and scaled up by simply adding additional hardware. Ceph can support up to exabyte range with beyond tens of billions of objects without compromising on performance, because Ceph components are designed to scale and perform linearly.
Ceph is also well known for and used for …
Software-defined storage
An SDS solution can be run on commodity and heterogeneous hardware solutions. Ceph decouples data from physical hardware using software abstraction layers. Thin provisioning of block storage is used for disk usage optimization. Tier-based storage solutions can be configured with Ceph for streaming, backup and archiving – for example, in the surveillance, media and entertainment industries.
Open source
Ceph is highly reliable, easy to manage, and free. Enterprise support can be purchased if it is required. It also runs on commodity hardware.
Cloud storage and storage backend
Ceph can be integrated with industry-leading cloud platforms including OpenStack, CloudStack, OpenShift and Kubernetes. It can also be used as a storage target for backup solutions: Commvault, NetApp Vault, Trilio, TrilioVault, Veeam, etc. In the cloud computing industry, Ceph is used to store workload and backup data.
Disaster recovery and high availability
A Ceph cluster runs on multiple disks over multiple servers. Ceph completely distributes data with no single point of failure. The Ceph Multisite feature allows data replication between multiple Ceph clusters. It contains multiple active zones (able to receive write operations) which are configured for disaster recovery. Each zone is backed by its own Ceph Storage Cluster. If one of the zones experiences a failure, another zone is there to provide disaster recovery.
In the IT datacenter industry, Ceph is well used for migration/moving of data between multiple distributed datacenters.
Data security
By configuring either replication or erasure coding technology at the software level, you can ensure that there will be no data loss if a hardware or network issue occurs. Ceph is self-healing; it detects and corrects failure without impacting data availability.
Massive data storage
Because of Ceph's ability to scale out while maintaining data integrity and availability, which is essential for handling large datasets, Ceph is often selected in the education sector, as well as the high performance computing and bioinformatics industries, for the following use cases:
- Store data for data-intensive applications, scientific computing, and big data analysis
- Store large volumes of data from simulation and research projects
- Share large volumes of data with massive storage solutions.
Pitfalls of Ceph
While Ceph can be a powerful solution to address modern storage requirements with its flexibility, configurability, and rich functionalities developed by thousands of developers in the open source community, its design, implementation, and operation can be challenging. There are many considerations which impact not only capacity, capability, availability, and density, but also performance.
There will be an unnecessary performance penalty if there is misconfiguration of cluster settings (e.g., setting faster data recovery might contribute to slow cluster performance). Ceph Logical Partitions need to be configured for workloads of different types.
One slow server in the Ceph cluster can also bring the performance of the whole cluster down. Servers, disks and connectivity should be carefully selected and designed for an optimal solution. Storage tiering, based on retention or performance, will be required in most cases.
In conclusion, comprehensive domain knowledge of hardware, networking and Ceph are needed for an optimal Ceph implementation which is suited to your workload requirements.
How HPE can help you build it
Hewlett Packard Enterprise has extensive offerings to accelerate the transformation of enterprise IT to become more open, flexible, agile, and innovative by leveraging modern software-defined storage solutions such as Ceph. HPE servers and storage are validated with the Ceph storage solution and certified with enterprise distributions of Ceph.
Reference architectures with HPE hardware for the canonical distribution of Ceph and the Red Hat distribution of Ceph are publicly available. HPE hardware is validated for specific use cases – for example, we offer Telco Blueprints for Telco implementations of OpenStack and Ceph. HPE ProLiant DL360 and DL380 are used in the HPE Telco Blueprints.
HPE Advisory & Professional Services has strong expertise and experience. The Global Cloud Everywhere Practice within HPE Advisory & Professional Services guides enterprises on their open-source-software-based cloud transformation journey with a proven methodology and vast experience/expertise.
Learn more about IT consulting services from HPE.
Services Experts
Hewlett Packard Enterprise
twitter.com/HPE_Services
linkedin.com/showcase/hpe-services/
hpe.com/services
ServicesExperts
HPE Services Team experts share their insights on the topics and technologies that matter most for your business.
