Reports & Publications

64 GPU AI Computing Performance Comparison Test H3C RoCE Network (S9827 Series Switches) vs. InfiniBand Network

Sponsor: New H3C Technologies Co., Ltd
H3C RoCE Network (S9827 Series Switches) vs. InfiniBand Network

Abstract

H3C S9827 switches, including the H3C S9827-128DH, H3C S9827-64EP, and H3C S9827-64E, support high-density 800GE/400GE/200GE ports with powerful forwarding capabilities. They can accommodate up to 64× 800GE or 128× 400GE ports while being compatible with LPO optical modules and ZR long-distance optical modules. Additionally, they support port splitting to 256× 200GE ports, offering extremely high port density and strong forwarding capabilities to meet the requirements of ultra-large data centers and AIGC computing power networks for high-density, non- blocking server access. Furthermore, the 400G QSFP112 ports are backward compatible with 200G QSFP56 and 100G QSFP28 optical modules.


Tolly conducted performance evaluations using the NVIDIA Collective Communication Library (NCCL) with 64 GPUs and tested the large language model Llama3 under different network architectures.  Specifically, the tests compared the performance differences between an RDMA over Converged Ethernet (RoCE) network using H3C S9827 switches and an InfiniBand (IB) network using NVIDIA QM9700 switches in a 64-GPU environment.

 

The results of the NCCL and Llama3 tests indicate that, under the same workload scenarios, RoCE delivers performance comparable to IB and provides a consistent user experience.