Reports & Publications

64 GPU AI Computing Performance Comparison Test H3C RoCE Network (S9825 Series Switch) vs. InfiniBand Network

Sponsor: New H3C Technologies Co., Ltd
H3C RoCE Network (S9825 Series Switch) vs. InfiniBand Network

Abstract

H3C S9825 Switch Series, including the S9825-64D and S9825-128B switch models, is a new-generation of high-performance and high-density 400G/200G/100G Ethernet switches developed by H3C for data centers. Designed with a single forwarding chip, the switch provides high-density 400GE/200GE/100GE ports that all support full line-speed forwarding. It also supports removable power supplies and fan trays in redundancy.

 

Tolly conducted tests evaluating performance of the NVIDIA Collective Communication Library (NCCL)  with 64 GPUs and the large-scale model Llama3 under different network architectures. Specifically, the tests compared the performance differences between an RDMA over Converged Ethernet (RoCE) network using H3C S9825 switches and an InfiniBand (IB) network using NVIDIA QM9700 switches in a 64-GPU environment.

 

The test results for NCCL and the large language model Llama3 indicate that RoCE delivers performance comparable to IB and provides a consistent user experience in the same workload scenarios.