Around the Storage Block
Showing results for 
Search instead for 
Did you mean: 

What Object Storage Brings to Big Data Environments


Elevating protection and retention to a new tier using low-cost object storage with an object storage deployment based on HPE Apollo hardware and Scality RING software.

Object storage_data center_blogs.jpgAre you finding that you're often running out of space on your Tier-1 storage?  Is it tough to run your business driving applications due to excessive growth? And are budget constraints prohibiting you to keep up with the growth? 

Here’s an idea: Maybe you need to offload workload from your Tier-1 disk to a lower-cost object storage solution. Here’s how we can help.

What is object storage and why should you be considering it?

Object storage manages data as objects, as opposed to other storage architectures like file systems which manage data as a file hierarchy, and block storage which manages data as blocks within sectors and tracks. Object storage has a few characteristics that differentiate it from ‘traditional’ storage. These are:

  • Lower cost (to make it feasible to store huge amounts of data at a sensible price point)
  • Resilience in the range of 11x9 along with geo-distribution options
  • Extremely space-efficient data redundancy schemes (lower cost) thru erasure coding
  • (Almost) unlimited scalability
  • Data storage consolidation that thereby eliminates boundaries and silo inefficiencies

Object storage use cases

Object storage solutions can be deployed to address a wide variety of your storage needs. Ultimately, object storage allows the movement of non-latency sensitive warm/cold data from valuable Tier 0/1 disk storage to a lower-cost tier. This table outlines many object storage uses cases including but not limited to big data frameworks and HPC archive storage, online and active archives, content distribution, web and cloud services, backup and recovery and petascale enterprise storage.Object Storage_use case_table.jpg
HPE Apollo/Scality RING

Now let’s look at one instance of an object storage deployment – in this case based on HPE Apollo hardware and Scality RING software (which we formally call HPE Scalable Object Storage with Scality RING).

HPE Scalable Object Storage is a software-defined storage (SDS) solution for petabyte-scale data storage. It creates scale-out storage systems that converge the storage of object-based and file-based data from multiple applications. The RING can seamlessly scale from a distributed system, atop a minimum cluster of six HPE Apollo servers to systems comprising hundreds of physical storage servers with a total storage capacity running into hundreds of petabytes.HPE Apollo Scality RING Connector DesignJ.jpg

Let’s look at two use cases in more detail:

One: Hadoop tiering

Once in place, Hadoop Clusters have a tendency to grow in data size sometimes at exponential rates. Performance from running analytics against the cluster is being impacted and in addition, the HDFS redundancy scheme makes you consume 3x the storage capacity. Keeping all data on the top tier disks, impacts analytics capabilities by limiting the amount of data that can be collected, kept, and analyzed. Ultimately, accuracy of the analysis might suffer. Also cluster operations become very time consuming. 

By offloading the archives onto a Cold storage tier, you can free up the Hot and Warm storage tiers for HPC processing. Now your replica count can be reduced to 2, with 1 replica stored on DISK and 1 replica archived to HPE Scalable Object Storage, as opposed to multiple replicas stored on the Hadoop Cluster’s DAS. Scality S3 BucketsJ.jpg

By deploying the HPE Scalable Object Storage solution for your Hadoop Cluster, you have almost unlimited scalability without adding more cost increases expanding your higher tiered disk.   By offloading to Object Storage, you have more accurate analytics on larger data sets, and faster cluster operations due to large data amounts being offloaded and managed by the Scalable Object Storage.  Most important, with the use of ARC erasure coding, you have a higher availability of your data than having it reside on the Hadoop Cluster. 

Two: Enterprise backup targeted at object storage

Deploying an HPE Scalable Object Storage solution in your backup enterprise, is another good use of storage tiering with Object Storage.  Backup environments are growing on the average of 30% a year, then add long term archive/legal hold compliance, and much growth becomes twofold by holding data indefinitely. Here at HPE, I work a lot with Spectrum Protect (formerly TSM) customers, who are typically large enterprise customers, managing several petabytes worth of backup data. Often times, when I am talking to them about how HPE StoreOnce is a good fit for their daily backup/recovery and disaster recovery needs, customers are insistent on also removing physical tape entirely out of TSM.  While HPE StoreOnce is a great fit for short term backup data, and disaster recovery, you wouldn’t want to put long term archive or legal hold backup data on it.  In the past, my conversations with the customer on this topic would veer to keeping physical tape for long term archive backup data, and often times this would frustrate the customer that there is no true tapeless solution for their backup/recovery and disaster recovery needs. 

Now a great alternative is to connect your backup servers to the Scality RING. For this purpose, I will focus on IBM TSM/Spectrum Protect. When you integrate Scality with the TSM server, you will connect to the Scality RING via NFS and can also use the S3 protocol to back up to the cloud. Since TSM environments tend to be large, they have demanding requirements in SLAs around RTOs & RPOs to backup/recover their data within a certain window. To help achieve these requirements, you can use a Scality Connector Node for each TSM storage pool you want to write to in the Scality RING. The more Connector Nodes you deploy, the better the performance will be between the TSM Server and the RING. Object Storage Solution OverviewJ.jpg

As I talked about in the beginning, think of object storage as part of a total solution you will be deploying in your data center. HPE StoreOnce is a great solution for your daily short term backup data, with HPE Scalable Object Storage better suited for long term archives that need to kept anywhere from one year to forever. 

You might have multiple sites that you would like to deploy HPE StoreOnce and the Scalable Object Storage solution to run local backups and archives, using StoreOnce and Scality to replicate between both sites, and/or backup to the Cloud. Or maybe, you would like to just use the Scalable Object Storage as your target for all TSM backup and archive operations, possibly extending the RING into the Cloud. 

In summary, solution delivers performance, scalability, cost effectiveness, and adaptability while satisfying compliance requirements of data retention and integrity.  HPE Apollo systems running Scality RING software is both designed and optimized for enterprise-scale data archives and cloud-based storage solutions.

Get going with a one-day workshop

HPE offers several options and solutions to meet your big data, cloud and backup needs. Contact your local HPE sales team to arrange for a presales one-day workshop to see how object storage can be part of the solution. We are happy to whiteboard an architecture tailored to your specific requirements—and help you better understand how the HPE Scalable Object Storage is an integral part of your IT infrastructure.

Learn more about HPE and Scality.

Cyrus Cama HPE Storage.jpg

 Meet Around the Storage Block blogger Cyrus Cama, Technical Marketing, HPE Storage.



About the Author


Our team of Hewlett Packard Enterprise storage experts helps you to dive deep into relevant infrastructure topics.

Starting June 23
HPE Discover Virtual Experience
Joins us for HPE Discover Virtual Experience live and on-demand
Read more
Online Expert Days - 2020
Visit this forum and get the schedules for online Expert Days where you can talk to HPE product experts, R&D and support team members and get answers...
Read more
View all