Surprisingly Useful AI Article Enhancements
AMD has unveiled its latest accelerators and networking solutions, aimed at supporting the growing demand for Ai infrastructure. The company announced the release of its AMD Instinct MI325X accelerators, the AMD Pensando Pollara 400 NIC, and the AMD Pensando Salina DPU. These products are designed to boost performance for data centres and generative Ai models.
Built on the CDNA 3 architecture, AMD Instinct MI325X accelerators are geared to handle demanding Ai tasks, from training foundation models to inferencing. This launch enables AMD’s partners, including Dell Technologies, HPE, Lenovo, and Supermicro, to deliver high-performance Ai solutions.
Forrest Norrod, Executive Vice President and General Manager of AMD’s Data Center Solutions Business Group, said, “AMD continues to deliver on our roadmap, offering customers the performance they need and the choice they want to bring Ai infrastructure to market faster.”
Memory and Compute Capacity
The AMD Instinct MI325X offers 256GB of HBM3E memory, supporting 6TB/s of bandwidth, which is 1.8 times more capacity and 1.3 times more bandwidth than its predecessor, the H2001. According to AMD, this leads to significant improvements in inference performance across various models. The MI325X can achieve up to 1.4 times the inference performance on Mixtral 8x7B at FP16 compared to the H200.
These accelerators are set for production in Q4 2024, with system availability expected from Q1 2025. Major platform providers such as Dell, Lenovo, and Supermicro are lined up to offer systems equipped with the new technology.
Looking ahead, AMD previewed the Instinct MI350 series accelerators, based on the CDNA 4 architecture. The MI350 series is expected to deliver a 35-fold improvement in inference performance and continue AMD’s leadership in memory capacity with up to 288GB of HBM3E per accelerator. The MI350 series is slated for release in the second half of 2025.
Advancements in Ai Networking
As part of its Ai infrastructure strategy, AMD introduced the Pensando Salina DPU and the Pensando Pollara 400 NIC. These components are designed to optimise Ai networking, split into front-end and back-end operations. The front-end, managed by the Salina DPU, delivers data to Ai clusters, while the back-end, managed by the Pollara 400 NIC, ensures efficient data transfer between accelerators.
The Pensando Salina DPU offers up to twice the performance, bandwidth, and scale of its previous generation, supporting 400G throughput for fast data transfers. Meanwhile, the Pollara 400 NIC is touted as the first UEC-ready Ai NIC, optimised for accelerator-to-accelerator communication and scalability in Ai networks.
Both products are expected to be available to customers by the first half of 2025.
AMD’s Ai Software Enhancements
In addition to hardware, AMD continues to enhance its software ecosystem, focusing on its ROCm open software stack. ROCm 6.2 now supports critical Ai features such as FP8 datatype and Flash Attention 3, which AMD claims offer up to a 2.4 times improvement in inference performance and 1.8 times improvement in training for various large language models.
AMD’s software efforts also extend to collaborations with popular Ai frameworks like PyTorch and Triton. These partnerships enable AMD compute engines to be used in widely adopted Ai models, including Stable Diffusion 3 and Meta’s Llama 3. The goal is to provide seamless performance with AMD Instinct accelerators.
This focus on both hardware and software advancements underscores AMD’s commitment to developing solutions that support the increasing demands of Ai infrastructure across industries.
Last Updated on October 14, 2024 by Nick Ross
