See how far supercomputing storage has come in 50 years

[UliPlechschmidt]

Understand the current state of supercomputing storage and how it can help you to address your storage challenges in this recap report from SC22.

SC22 was one for the history books! For one, it was the first time the community met fully in person since 2019. But for HPE, it was extra special because we celebrated the 50^th anniversary of Cray.  

The most popular photo spot of the whole conference was the Cray-1 supercomputer at the HPE booth.

The whole supercomputing industry really started with the Cray-1 and while there are many Cray-1s in museums around the world — like serial number (SN) 11 in the Science Museum London, SN 14 in the Smithsonian National Air and Space Museum in Washington D.C. or SN 26 in the German Museum in Munich, the machine in the HPE booth is special because it has the serial number SN1. This very first Cray-1 was installed in the year 1976 at Los Alamos National Laboratory as the most powerful supercomputer on the planet with the following — by then earth-shattering — specifications:

• Compute performance: 160 megaflops
• Storage capacity: 303 megabytes

As this is a storage-focused blog, let’s have a look at the incredible improvements that have happened since then on the supercomputing storage side.

How far supercomputing storage has come in 50 years

To match the storage capacity available to today’s most powerful supercomputer — the HPE Cray EX-based Frontier supercomputer at the Oak Ridge Leadership Computing Facility — with the Cray ClusterStor E1000-based Orion storage system, you would need to line up Cray-1 supercomputers side-by-side 145 times around the earth.

Unbelievable, right? In 1976 everybody would have said that would be utterly impossible. Just imagine, what innovations the next 50 years will bring!

Let’s have a look at the current state of supercomputing storage using the November 2022 Top500 list that was released on the first day of the SC22 conference in Dallas.

If we look at the Top 100 most powerful supercomputers on that list from an external storage perspective there are couple interesting insights.

Lustre continues to dominate in the Top 100 supercomputers

This graphic illustrates the trend in the Top 100 in the last years with Lustre-based storage systems being attached to roughly 80% of supercomputers and IBM Spectrum Scale-based systems supporting roughly 20% of machines.

The higher up in the Top 100 you look, the higher the share of Lustre-based storage systems. The analysis below illustrates the external storage systems of the Top 20 by vendor. Of those top 20, 17 use Lustre and three are connected to IBM Spectrum Scale-based storage systems.

Due to its open-source nature, Lustre is more often used in government-funded organizations than in enterprises. In contrast, IBM Spectrum Scale is more often deployed in enterprises due to its very rich set of enterprise storage features. The higher you get in the Top 500 list, the more government-funded organizations you find that provide services on a national level like national labs or weather forecasting centers.

As the #1 HPC¹ vendor, we strive to provide complete end-to-end solutions to all organizations that are using modeling & simulation (mod/sim), artificial intelligence (AI/ML) and high performance data analytics (HPDA) to stay ahead in their segments.

This is why we offer a complete portfolio of parallel storage solutions (see below).

When we talk about supercomputing storage, we do not just mean Lustre. For example the #79 system on the current Top500 list — the alpha ONE AI supercomputer at Aleph Alpha — features IBM Spectrum Scale-based HPE Parallel File System Storage as part of the end-to-end HPE Machine Learning Development System.

If you’re using HPE servers like HPE Apollo systems for MOD/SIM, AI/ML, or HPDA workloads today but have non-HPE file storage attached to them for historical reasons, know that we can provide you better price/performance with the additional value of having your full stack under support from HPE Pointnext Services. No more vendor finger pointing!  Even better, this statement is independent of what business you are in and what the size of your organization is. And independent of if you want to run your end-to-end system with own team in your premises or prefer to have us run it as your fully managed private cloud in your premises or in another data center location near you.

Contact your HPE representative today to learn more. To dig deeper right away:

Download the paper: Spend less on HPC/AI storage and more on CPU/GPU compute

Find more details online: HPC and AI storage

Uli Plechschmidt
Hewlett Packard Enterprise

twitter.com/hpe_hpc
linkedin.com/showcase/hpe-ai/
hpe.com/us/en/solutions/hpc

¹ Hyperion Research, Pre-SC22 HPC Market Update, November 2022