The HPE Cray XD670 system is #1 in natural language processing and a top performer across all MLPerf Inference v4.0 benchmark models where it participated.
New AI benchmark results are out from MLCommons and the HPE Cray XD670 powered by NVIDIA GPUs has achieved the #1 spot for natural language processing (NLP with Bert 99.0 Offline scenario). This system, which powers one of the largest AI clouds in Europe, was also a top performer in all the MLPerf Inference v4.0 benchmark categories it participated in, including GenAI, computer vision and large language models.
If you are an AI service provider, or if your organization is already past the initial AI piloting phase and you are building models and extending AI across your enterprise, you will greatly benefit from the scale and power of the HPE Cray XD670 system.
The HPE Cray XD670 is specifically designed and optimized for AI workloads that are heavily parallelized, requiring GPU acceleration for optimum performance. Examples include large language model training, tuning and inference, natural language processing, and multi-modal training. The system features eight NVIDIA® H100 Tensor Core SXM5 GPUs and the latest advances in hardware and software to deliver a complete, scalable solution, purpose-built for AI.
- Natural language processing performance. #1 top performing server for NLP: The Bert NLP model is widely used for question and answering and language inference.1
Superior AI performance across the board in industry-standard benchmarks
Unbiased benchmarks are one of the criteria customers can use when evaluating different technologies to deploy AI applications. MLCommons is a leading AI engineering consortium built on a philosophy of open collaboration to improve AI systems. They"build AI benchmarks and have just announced new results from the MLPerf Inference v4.0 benchmark suite which, "delivers industry standard machine learning (ML) system performance benchmarking in an architecture-neutral, representative, and reproducible manner."2
HPE successfully submitted results demonstrating a range of high-performing inference systems for the datacenter on natural language processing (NLP), generative AI (GenAI), large language models (LLM) and Computer Vision (CV). HPE performance results included offerings from the HPE Cray and HPE ProLiant server families and were part of the Datacenter-Closed, Datacenter-Open, and Datacenter-Network divisions. To get the details and view the MLPerf Inference v4.0 benchmark results visit the Inference benchmark page in the MLCommons website.
In addition to the #1 spot in NLP, the HPE Cray XD670, featuring eight NVIDIA H100 Tensor Core GPUs SXM with 80 GB of memory, was a top performer across all the models where it participated, including GenAI, CV and LLMs. These leading results on a range of AI benchmarks demonstrate the system’s superior performance for AI environments. Let’s explore more details:
- Large language models. Llama 2 70B performance: Llama 2 is a 70 billion parameter model popularly used for chat and dialogue use cases.3
- Generative AI. Stable Diffusion-XL performance: SDXL is a latent diffusion model for text-to-image synthesis.4
Flexibility and scale for demanding AI workloads
HPE Cray XD670 is compatible with a broad range of technologies, including accelerators, storage, networking, power, and cooling options giving you flexibility in an open ecosystem.
With an extremely dense configuration, your environment can scale with your business needs. High-performance storage is carefully integrated with the HPE Cray XD670 creating a unified architecture that can scale rapidly to overcome the biggest data center challenges. Optional plug-and-play direct liquid cooling brings power efficiency and enables energy re-use helping you advance your sustainability goals. To round out the solution, HPE offers an extensive software portfolio to help you streamline the development of data-driven applications and launch them into production.
Advancing AI initiatives
As you advance your AI projects, there are many decisions to make along the way. A fundamental one is your choice of technology partner. HPE and NVIDIA are well positioned to help enterprises and AI service providers at all the stages of the AI journey — from preparing data, to piloting projects to getting productive and extending AI across the enterprise. Through HPE AI Services, we offer expertise to help you explore, experiment, develop and evolve AI to address the individual needs of your organization, no matter its size.
Part of our broad AI portfolio, the HPE Cray XD670 system is optimized for LLM training, tuning and inference, NLP, and multimodal training. Recent benchmark results can give you the confidence this platform delivers superior performance across a range of AI workloads.
Discover more
Visit the website to learn more about the HPE Cray XD670.
For all the details on the benchmarks and results visit the MLCommons website or the MLCommons Inference v4.0 press release.
Diana has spent the past 25 years working with the technologies that power the world’s most demanding IT environments and is interested in how solutions based on those technologies impact the business and the world. A native of Colombia, Diana holds an MBA from Georgetown University and has held a variety of regional and global roles with HPE in the US, the UK and Sweden. She is based in Stockholm, Sweden. Connect with Diana on LinkedIn.
1. Bert NLP model: https://arxiv.org/abs/1810.04805
2. MLCommons, New MLPerf Inference Benchmark Results Highlight The Rapid Growth of Generative AI Models, March 27 2024
3. Llama2 70B model: https://arxiv.org/abs/2307.09288
4. Stable Diffusion-XL model: https://arxiv.org/abs/2307.01952
ComputeExperts
Our team of Hewlett Packard Enterprise server experts helps you to dive deep into relevant infrastructure topics.