Servers & Systems: The Right Compute

Comparing HPE on-premises infrastructure vs. Amazon Web Services (AWS): Price-performance analysis


HPE vs. AWS Blog 4 Blog.jpgPart 4: We continue examining how on-prem infrastructure compares to Amazon Web Services (AWS) by putting TCO and workload throughput together for a price-performance analysis.

In Part 3 of this series, Master Technologist Dr. Paul Cao provided an overview of the cloud-scale advanced analytics workload we used in comparing HPE on-prem vs. AWS and he reviewed the detailed performance results from each configuration.  In Part 4 of this blog series, we combine the TCO data from Part 2 with the throughput results discussed in Part 3 to yield a price-performance metric for each of the solutions evaluated in the study. You can download the complete technical paper here: HPE On-Prem vs. AWS. (Registration is required.)

Price-performance ratio

The familiar term “bang for your buck” is most commonly attributed to the 1950s U.S. Defense Secretary Charles Wilson who used it to indicate the relative ordinance explosive power per dollar cost. Today, this has become a universal idiom used more generally to describe relative value per a normalized unit of cost. That is to say, higher is better, as in “more bang for your buck.” In the world of IT, the analogous term is “the price-performance ratio,” or simply, price-performance, which describes how many dollars it costs to achieve some normalized unit of work. I.e. lower is better.

Cost and performance, each by themselves, tell a portion of the story and must be combined together for a true value analysis. Evaluating and selecting compute infrastructure solely based on the lowest unit-cost of the components might lead to purchasing many more units, interconnect, licenses which increase the cost to achieve a required aggregate workload throughput level. Similarly, selecting infrastructure solely based on the highest unit performance, may not be the best alternative either.

Understanding the range of performance service-levels that are required along with the relative costs to achieve those service-levels for all options that are being considered is the key to making the right business decision.

Price-performance of HPE on-prem vs. AWS 

HPE vs AWS - Blog 4 - Pict 1.pngThe calculation of the price-performance ratio is done simply by taking the TCO of a complete configuration and dividing it by the performance throughput for that same configuration. The cost is in dollars and, for this workload, the performance is measured in Queries per minute (Qpm) yielding a price-performance metric in $/Qpm. The lower the price-performance, the better the value. As we saw in Parts 2 and 3 of our blog series, the TCO for the HPE on-prem solutions were significantly lower than the AWS solutions while the HPE performance registered significantly higher than AWS. (You know where this is going. . . ) 

Figure 1 lays out the price performance as measured for the five configurations evaluated in the study. For each AWS configuration we include the measured price-performance for each of the purchase options, 3-Yr reserved, 1-Yr reserved and on demand. 

In comparing the HPE Gen10 server solution to the AWS m5, with a 3-Yr amortized payment schedule, the monthly price-performance of the AWS m5.24xlarge configuration is 2.5 times higher than HPE Gen10.  Simply stated, for a specific infrastructure throughput requirement, you pay 2.5 times more each month to achieve a specified workload throughput with the AWS m5 infrastructure than you would with the HPE Gen10 infrastructure.  If the same AWS m5 infrastructure is purchased as on-demand, the cost is 5 times more each month than the HPE Gen10 on-premises infrastructure. The Broadwell comparison results tell a similar story.

Generation-to-generation comparisons

When comparing the HPE Gen9 to the HPE Gen10, the gen-to-gen price-performance improves by 13.7%. This is in line with decades of generational performance improvements. In the AWS gen-to-gen comparison going from m4 to m5, each with the 3-Yr reserved purchase option, the price-performance only improves by 4.8%. It is not clear why the AWS gen–to-gen price-performance improvement is lower than historical on premises improvements. Any attribution on our part would be speculation. Also note that with the 1-Yr reserved and the on-demand purchase options, the AWS gen-to-gen price-performance is even lower at 0.5% and 3.2%, respectively.

IOPS-provisioned drives

For Hadoop implementations, while not entirely necessary, we typically employ the use of solid-state disks (SSDs) to store temp data which has a measured high IOPS requirement for this workload. The aggregate performance increase for the entire on-prem configuration is in the range of single-digit percentages; and the addition of the SSDs raises the configuration TCO by less than 1%. In this case, it makes business sense to use such SSD drives for temp storage as it lowers the price-performance of the entire configuration by a few percent.

The range of instances offered through AWS do not contain combinations of elastic block storage (EBS) and local SSDs for storage. Each offers either local storage or EBS storage, but not both. So, we couldn’t simply throw an SSD into the m4 and m5 instances to accelerate temp data. Instead, we configured equivalent EBS drives for the temp data specifying an EBS 20,000 IOPS-provisioned drive (io1) for each of the m4 and the m5 configurations’ nine worker instances. After the testing was completed we were quite surprised to learn that the cost of this drive, included in each of the nine worker instances, increased the monthly TCO for the AWS m5 configuration by 57% or $12,240 per month. (Whoops!)  

Since it would have unnecessarily disadvantaged the TCO of the AWS EBS-based comparisons, we removed the cost of these drives from the TCO and price-performance calculations in all of the examples of this study.

In summary, all performance results in the m4 and m5 configurations were obtained using the high IOPS EBS drive, but we removed the cost of the drives in all TCO and price-performance comparisons for this study.

Control over performance, privacy and security

The results of this study illustrate a fairly significant difference in the aggregate throughput measurements for on-prem and public cloud infrastructure deployments. These differences exist in the comparisons for both the Skylake and Broadwell generations. We’ve also shown how that impacts the pocketbook beyond the price of the components.

With such similar infrastructure specifications, why would the public cloud implementation be so much lower in performance? It boils down to the amount of control we have over the underlying infrastructure and its architecture. With on-prem systems, physical access allows us to optimize the physical memory, balance the IO across CPU sockets and PCIe, specify multiple backplanes for drive placement, control the network architecture and manage the platform’s policies for power efficiency, environment and other policies through direct access and control of the platform firmware. With the public cloud model, these control points are omitted in favor of the simplified as-a-service experience.

Price-performance is the primary topic of this study and similar detailed comparisons between on-prem and public cloud are not abundant in the public domain. Much more is written about transparency and control over privacy, security and data sovereignty and the challenges of these concerns in public cloud deployments. We include only a brief overview of these topics in the paper for completeness and provide a few links for a deeper reference to these topics. You might also want to read the recent Forbes article published by Moor Insights and Strategy on the evolving enterprise calculus of public cloud versus private infrastructure. The article references the HPE On-Prem vs. AWS technical paper while providing further perspectives on cloud infrastructure capabilities, control and cost.

Watch for the final blog in our on-prem infrastructure vs. AWS series

Some are willing to pay significantly more to achieve the benefits of the cloud consumption experience via public cloud for the reasons we have discussed in the series. In Part 5, the final part of this series, I discuss the rapidly narrowing gap of pay-per-use consumption alternatives between AWS public cloud and HPE on-prem infrastructure with GreenLake Flex Capacity.

Follow the blog series

In this blog series, we present details and insights around the HPE and AWS comparison for the following topics:

Part 1: Introducing the HPE on-prem vs. AWS primer

  • An overview of the study and the summary of findings in the comparison of HPE and AWS
  • An AWS primer to provide a brief overview of what is available in AWS EC2 IaaS capabilities

Part 2: Total Cost of Ownership (TCO)

  • The configuration options and selected configurations: HPE and AWS
  • The “all-in” cost analysis for the on-prem configuration, including costs for maintenance labor, data center infrastructure, energy and cooling, carbon footprint and warranty
  • The cost analysis for the AWS configurations based on reasonable configurations and purchase options

Part 3: The workload and throughput (performance) 

  • Overview of the cloud-scale advanced analytics workload
  • The throughput measurements for each configuration in total Queries per Minute (Qpm)
  • Analysis of each architecture and comparison of Local SSDs and EBS

Part 4: Price-performance and control

  • Price-performance for each configuration
  • A look at the cost of high-throughput EBS volumes
  • Analysis of ability to control attributes of performance, data sovereignty, privacy, security

Part 5: Elastic IT experience

  • Pay per use with variable payments based on actual metered usage
  • Dynamic and instant growth flexibility
  • Onsite extra capacity buffer

Lou Gagliardi.jpgMeet Infrastructure Insights blogger:
Lou Gagliardi, Sr. Lab Director, Enterprise Solutions & Performance, Hybrid IT, HPE
Lou joined Compaq Computer Corporation in 1988 and has held executive-level positions in server and storage development engineering and WW presales while at Compaq, Hewlett-Packard, Dell, Newisys and Spansion. He returned to HP in 2010 as Sr. Director of Integrated System Test.  In 2013, he transitioned to his current role to deepen HPE’s understanding of the character of the New Style IT workloads.




About the Author


Our team of HPE and other technology experts shares insights about relevant topics related to artificial intelligence, data analytics, IoT, and telco.