Advantage EX

What CIOs need to know about hybrid HPC


Hybrid HPC helps enterprises strengthen their digital transformation by allowing them to scale up massive resource sets, enhance security, flexibly develop infrastructure, and optimize costs.

Strengthen your digital transformation with hybrid HPC.Strengthen your digital transformation with hybrid HPC.As an enterprise CIO, you are likely caught between the twin forces of burgeoning workloads and limited resources. And just as likely, you may be considering hybrid HPC (high-performance computing) environments to support next-generation digital business models.

HPC has been around for several decades, primarily in the domain of advanced scientific research. At the same time, hybrid cloud computing environments have emerged as the go-to solution for organizations looking to streamline data infrastructure. As HPE's Bill Mannel noted last year in HPCwire, bringing the two together is a natural solution, as enterprise applications and services become more dependent on the heavy workloads associated with big data and advanced AI-driven analytics. Here are a few key considerations for CIOs embarking on a hybrid HPC journey.

Scale and control are equally balanced

According to Markets and Markets, the global HPC market is set to reach nearly $45 billion by 2022. One of the chief value propositions of hybrid HPC is that it allows you to scale up massive resource sets without giving up control of infrastructure. Many emerging workloads require highly customized support, which generally lacks all-cloud solutions, but would be prohibitively expensive to deploy strictly in house. With a hybrid solution, you can leverage the vast scale of the cloud to burst excess workloads onto third-party infrastructure while reserving the highly specialized work for tailored solutions in the data center.

This ability to customize hybrid HPC solutions offers a number of other key benefits, as well. For one, it extends familiar work environments into the cloud. This allows you to avoid the time, expense, and lost productivity involved in training data scientists and other knowledge workers on the intricacies of a given cloud provider's solution—some of which are built around the provider's proprietary framework. In a hybrid environment, the provider delivers basic resources at scale, while you decide how deeply you want to integrate processes with third-party infrastructure.

Users gain control over security, IaaS, and PaaS

Hybrid solutions also provide enhanced levels of security and compliance. Private and hybrid clouds can support HPC environments that are purpose built for multitenancy, which allows multiple users to share the same physical environment while maintaining complete isolation of data. In the public cloud, these options are usually limited. Multitenancy is particularly important to highly regulated industries like finance and healthcare, but it also provides an added layer of protection for applications like ecommerce and social media, which often contain sensitive information within their data streams.

Hybrid infrastructure also provides many opportunities to leverage infrastructure-as-a-service (IaaS) and platform-as-a-service (PaaS) architectures for a wide variety of use cases. IaaS, for example, delivers previously hardware-centric resources like compute, storage, and networking as programmable services, giving developers more control over the application stack to determine consumption levels, workload isolation, and other parameters. PaaS takes this functionality to the next level by preprovisioning operating systems, abstracted middleware and security/governance policies to cut down on provisioning time and application deployment.

Performance, cost, and hosting options are key

Some may wonder why the enterprise should bother with a hybrid solution when the public cloud already provides all the scale needed for such heavy workloads. While the public cloud does offer some advantages over private and hybrid solutions—namely, the ability to support linear scaling with parallel execution and offload many support and provisioning responsibilities—its drawbacks start to mount as workloads become more complex. For one thing, application performance can be temperamental due to network latency, poor memory allocation, restrictive SLAs, improper mapping, and the list goes on. ISV licensing costs can also quickly get out of hand with HPC-sized projects, and issues like data mobility, mapping, and fine-tuning of workloads to resources can be difficult when working with third-party infrastructure.

A hybrid environment can be hosted on local resources, at an off-site facility, or a mixture of both—either as a managed service or a distributed private cloud. This means it can be developed along a wide variety of capex and opex models, all while preserving the scale, resource consumption, and overall performance required of highly specialized workloads.

Successful architecture must be set up by design

But what, exactly, are some of the design criteria that CIOs need to consider when devising a hybrid high-performance computing solution? There's no one right answer, of course. In order to gain the maximum flexibility possible, however, the architecture should encompass several key elements:

  • Cutting-edge memory, storage, and acceleration—You'll need these in order to accommodate the broad spectrum of workloads the environment is likely to host.
  • On-demand provisioning—Easy access to high-performance, bare-metal clusters is crucial, preferably in ways that you can tailor to unique applications.
  • Multiple service entry points—These should include application-level, platform-level, and infrastructure-level options to allow users to engage the system as needs dictate.
  • Variable cost-consumption models—Resources are utilized in different ways by different jobs, so what works for one project may not work for another.
  • State-of-the-art security—Ideally, this should be comparable to best-in-class on-premises clusters.

In the final analysis, a hybrid solution should be purpose built to deliver a cloud-like experience to both administrators and end users. By leveraging the best in both local and cloud infrastructure, a hybrid environment offers the highest degree of flexibility and operational efficiency to take on the heavy workloads of today's digital economy.

Learn more about hybrid HPC

Data loads are not getting any smaller, while budgets, in most cases, are not getting any larger. In order to meet the time-honored mandate to do more with less, you need to not only embrace all available data and infrastructure technologies at your disposal, but also do so in a coordinated fashion that stresses maximum efficiency and flexibility. As many CIOs have already learned with traditional production workloads, hybrid solutions offer the right balance between cost and performance, with a wide variety of deployment modules capable of suiting virtually any project.

For more information on hybrid high-performance computing architectures and how they can help you manage the transition to a digital services model, check out the white paper on Hybrid HPC Cloud.

Meet InfraArthur Cole.pngstructure Experts blogger and freelance content developer Arthur Cole. Arthur is a longtime industry journalist with more than 25 years of experience covering high-tech fields, ranging from professional audio and video to IT infrastructure, cloud, telecommunications, and B2B applications and services.


Infrastructure Insights
Hewlett Packard Enterprise

twitter.gif @HPE_Servers


0 Kudos
About the Author


Our team of HPE and other technology experts shares insights about relevant topics related to artificial intelligence, data analytics, IoT, and telco.

HPE Webinars
Find out about the latest live broadcasts and on-demand webinars
Read more
Online Expert Days
Visit this forum and get the schedules for online Expert Days where you can talk to HPE product experts, R&D and support team members and get answers...
Read more
View all