Nvidia Spectrum-X: a network solution for generative AI with a 51 Tb/s switch

Generative AI has been a huge topic for the past six months. However, its training requires tens of thousands of high-performance GPUs, and solid hardware is also required for operation. But in order for this to work well, these computers also need to be connected, also with the possibility of achieving very high transfer speeds. Company Nvidia so for these purposes it brings not only a powerful GPU, but also a network solution Spectrum-X. It consists of a Spectrum-4 switch and a BlueField-3 DPU. This combination should offer 70% more performance for AI systems due to faster network connection. The chip for the Nvidia Spectrum-4 is produced by the TSMC 4N process, has dimensions of 90×90 mm and consumes 500 W.

A supercomputer, for example, will be built on this system Israel-1 aimed specifically at generative AI. It will use Dell PowerEdge XE968 servers based on the Nvidia HGX H100 platform with 8 GPUs. The Spectrum-4 is an Ethernet switch that can achieve an extremely high total transfer rate of 51 Tb/s. Together with Nvidia LinkX optics, it is thus possible to create networks with many 400GbE connections (or other speeds).

The speed is so high that in the version of the SN5600 switch (2U size) with 256 ports, all of them can be at 200GbE speed. If you need an even higher 400GbE speed, you get a maximum of 128 ports, and in the case of 800GbE, then a still high 64 ports. All this in one single switch. In case of leaf-spine topology, up to 16000 ports can be created.