Latest MLPerf Results: NVIDIA H100 GPUs Ride to the Top

Starting in 2019, NVIDIA is a regular on the calendar year list of the industry standard AI inference test MLPerf. Currently, NVIDIA AI is the only platform capable of running all MLPerf inference workloads and scenarios in both data centre and edge computing. Recently, NVIDIA H100 GPUs made their MLPerf debut, setting world records in all workload inference and outperforming previous generation GPUs by 4.5x.

NVIDIA's Journey With MLPerf

H100 Sets New Benchmark in All Data Centre Loads

In the latest MLPerf test results, H100 (aka Hopper) achieved the best single-accelerator performance of all six neural networks, demonstrating throughput and speed leadership in both single-server and offline scenarios. h100 sets a new benchmark across all workloads in the data centre category.

And compared to itself, the NVIDIA Hopper architecture outperformed the NVIDIA Ampere architecture by a factor of 4.5, although the Ampere architecture GPUs continued to lead across the board in the MLPerf results.

Of these typical AI training models, BERT is one of the largest and most performance-demanding of the MLPerf AI models, and Hopper's outstanding performance on the BERT model is due in part to its Transformer Engine.

These test results suggest that Hopper will be the optimal choice for those who need the highest performance on advanced AI models.

A100 Continues to Show Leadership

In addition to the H100 GPU, NVIDIA's A100 GPU continues to show leadership across the board in mainstream AI inference performance. In both data centre and edge computing categories and scenarios, the A100 GPU won more test entries than any other submitted result.

Since its debut at MLPerf in July 2020, the A100 GPU's performance has improved by a factor of 6 due to continuous improvements in NVIDIA AI software. The A100 achieved an all-around lead in the MLPerf training benchmark back in June, and the NVIDIA A100 also continued its single-chip leadership, proving to be the fastest in six of the eight tests.

The NVIDIA A100 Tensor Core GPU and the NVIDIA Jetson AGX Orin module for AI robotics demonstrated overall leading inference performance in all MLPerf tests, including image and speech recognition natural language processing and recommender systems.

The A100 GPUs are currently being used by major cloud providers and system manufacturers.

Orin Is the Winner in the Low-power Chip Test

In edge computing, NVIDIA Orin ran all MLPerf benchmarks and won the most tests of any low-power system-on-a-chip (SOC). And it was 50% more energy efficient than its debut at MLPerf in April.

In the last round of benchmarks, Orin ran five times faster and was two times more energy efficient than the previous generation Jetson AGX Xavier module.

Orin integrates NVIDIA Ampere architecture GPUs and powerful Arm CPU cores into a single chip. Hyperion), medical device platforms (Clara Holoscan) and robotics platforms (Isaac).

MLPerf Becomes A "Touchstone" for AI Applications

Users often employ many different types of neural networks in real-world applications. For example, an AI application may need to understand a user's voice request, classify an image, make a suggestion, and then provide a response with a human voice as a voice message. Each of these steps requires the use of a different type of AI model. As a result, users need GPU chips with general-purpose performance to handle them.

The MLPerf benchmark tests cover all these and other popular AI workloads and scenarios, such as computer vision, natural language processing, recommendation systems, speech recognition and more. These tests ensure that users will receive reliable and flexible deployment performance.

The results of previous years of MLPerf testing show that NVIDIA AI is supported by the industry's broadest machine learning ecosystem. In this round of benchmarking, more than 70 submissions ran on NVIDIA platforms. For example, Microsoft Azure submitted results running NVIDIA AI on its cloud service. In addition, 19 NVIDIA-certified systems from 10 system manufacturers participated in this round of benchmarking, including ASUS, Dell Technologies, Fujitsu, Gigabyte, Wisers, Lenovo, and Supermicro. Their results show that users can achieve outstanding performance with NVIDIA AI, whether in the cloud or in servers running in their own data centres.

With its transparency and objectivity, MLPerf empowers users to make informed buying decisions. The benchmark is widely supported by companies including Amazon, Arm, Baidu, Google, Harvard, Intel, Meta, Microsoft, Stanford and the University of Toronto. Nvidia's partners participate in MLPerf because they know it is an essential tool for evaluating AI platforms and vendors for their customers. The latest round of results shows that the performance they currently provide to users will grow as NVIDIA platforms evolve.

All of the software used for these tests is available from the MLPerf library, so anyone can access these world-class results, and optimizations are being added to NGC (NVIDIA's GPU acceleration software catalogue) in a steady stream of containerized form. There's also NVIDIA TensorRT, which was used to optimize AI inference for every submission in this round of testing.

Related News

1、Chip Packaging Lead Time Has Grown to 50 Weeks

2、Eight Internet of Things (IoT) Trends for 2022

3、Demand for Automotive Chips Will Surge 300%

4、Volkswagen CFO: Chip Supply Shortage Will Continue Until 2024

5、BMW CEO: The Car Chip Problem Will Not Be Solved Until 2023

6、Shenzhen: This Year Will Focus on Promoting SMIC and CR Micro 12-inch Project