Meet LUMI: accelerating pan-European innovation with HPE Cray architecture

[ComputeExperts]

LUMI, Europe’s fastest supercomputer, is driving advancements in scientific simulations, AI, large language model (LLM) development, and big data analysis. A pan-European endeavor backed by 11 countries and the EuroHPC Joint Undertaking (EuroHPC JU), the LUMI system has big responsibilities — and the resume to meet them.

Powered by the HPE Cray EX supercomputer, LUMI is the fifth fastest supercomputer in the world[i] and one of the most advanced AI platforms globally. With a sustained computing power of 380 petaflops, it equates to the combined performance of 1.5 million of the latest laptops[ii].

The supercomputer has dual tasks: addressing humanity’s toughest challenges while also boosting European research, employment, competitiveness, and innovation. And it’s tackling them. LUMI is driving breakthroughs in areas ranging from climate science and sub-atomic structures to genomics and quantum physics.  The system is available for researchers, private and public organizations, as well as startups and general subject matter experts.

LUMI, which stands for Large Unified Modern Infrastructure (LUMI), is funded by the European Union and a consortium of 10 countries:  Belgium, the Czech Republic, Denmark, Estonia, Iceland, Norway, Poland, Sweden, Switzerland, the Netherlands, and Finland. The system is hosted by CSC – IT Center for Science and located at a cutting-edge data center in Kajaani, Finland.

Empowering advancements in healthcare, sustainability, and beyond

LUMI's AI capabilities enable neural networks to detect cancer early and quickly simulate drug efficacy. With such a tool, pathologists can more accurately diagnose cancer growth and simulate patient responses to various treatments. The outcome will be the most personalized care possible as quickly as possible.

LUMI also supports the Destination Earth initiative, helping researchers gain the insights required to adapt to and potentially alter the course of climate change. Scientists can now simulate Earth's ecosystem down to 5 kilometers—something previously impossible.

Large LUMI-built computer model ensembles are being used to model Antarctic ice flows, helping answer crucial questions about how much the Antarctic Ice Sheet will contribute to rising sea levels in the coming centuries. For example, it will be possible to predict the likelihood of sea levels rising by 1 meter by 2100 if fossil fuel consumption remains high. Such estimates are vital for policymakers. The scale of LUMI allows researchers to run the computing model hundreds of times, each new simulation providing a small step toward quantifying Antarctica’s impact on future sea levels.

The engine behind the magic

LUMI is based on an HPE Cray Supercomputing EX supercomputer with next-generation AMD EPYC™ processors and AMD Instinct™ accelerators. It takes up almost 400m² of space and weighs nearly 150,000 kg.

The GPU partition consists of 2,978 nodes. Each node has 64-core AMD Trento CPU and four AMD MI250X GPUs. The total number of AMD GPUs is 11 912. Also, each node has four 200 Gb/s network interconnect cards. It means its injection bandwidth is 800 Gb/s. A single MI250X card can deliver 42.2 TFLOP/s of performance in the HPL benchmarks[1].

The GPUs are not the only partition in LUMI. Another one is LUMI-C which uses CPU-only nodes. Each node has 64-core 3rd-generation AMD EPYC™ CPUs and min. 256 GB and max. 1024 GB of memory.

In total there are 2,048 dual-socket CPU nodes. It means it uses 4,096 AMD Milan CPUs, over 262,000 CPU cores. The system also has a partition with large memory nodes. The partition has 32 TB of memory. For workload visualization LUMI uses 64 Nvidia A40 GPUs.

LUMI’s storage system consists of three components. The first one is a 10 PB all-flash Lustre system. It is used for short-term fast access. The second one is a longer-term, more traditional 80 PB Lustre system featuring a traditional HDD. For easy data sharing and project lifetime storage LUMI has 30 PB of Ceph-based storage. The partitions—compute and storage—are connected to the fast HPE Slingshot interconnect of 200 Gb/s.

According to the Green500 list (published in June 2024, LUMI is the 12th greenest supercomputer on the planet. It uses 100% hydro-powered energy, and its waste heat is used to heat hundreds of households in the city of Kajaani.

---

Compute Experts
Hewlett Packard Enterprise

twitter.com/HPE_Cray
linkedin.com/showcase/hpe-servers-and-systems/
hpe.com/supercomputing

[i] Source: https://top500.org/lists/top500/list/2024/06/

[ii] Source: https://www.lumi-supercomputer.eu/lumi-again-among-the-fastest-and-greenest-supercomputers-in-the-world/