Dan's Biz Bookshelf: 'Revenge of the Tipping Point'
Marcy's Musings: Engineering Economics
Punching Out: Should You Sell Your Company to a Private Equity Firm?
NVIDIA Blackwell's High Power Consumption Drives Cooling Demands; Liquid Cooling Penetration Expected to Reach 10% by Late 2024
July 30, 2024 | TrendForceEstimated reading time: 2 minutes
With the growing demand for high-speed computing, more effective cooling solutions for AI servers are gaining significant attention. TrendForce's latest report on AI servers reveals that NVIDIA is set to launch its next-generation Blackwell platform by the end of 2024. Major CSPs are expected to start building AI server data centers based on this new platform, potentially driving the penetration rate of liquid cooling solutions to 10%.
Air and liquid cooling systems to meet higher cooling demands
TrendForce reports that the NVIDIA Blackwell platform will officially launch in 2025, replacing the current Hopper platform and becoming the dominant solution for NVIDIA's high-end GPUs, accounting for nearly 83% of all high-end products. High-performance AI server models like the B200 and GB200 are designed for maximum efficiency, with individual GPUs consuming over 1,000W. HGX models will house 8 GPUs each, while NVL models will support 36 or 72 GPUs per rack, significantly boosting the growth of the liquid cooling supply chain for AI servers.
TrendForce highlights the increasing TDP of server chips, with the B200 chip's TDP reaching 1,000W, making traditional air cooling solutions inadequate. The TDP of the GB200 NVL36 and NVL72 complete rack systems is projected to reach 70kW and nearly 140kW, respectively, necessitating advanced liquid cooling solutions for effective heat management.
TrendForce observes that the GB200 NVL36 architecture will initially utilize a combination of air and liquid cooling solutions, while the NVL72, due to higher cooling demands, will primarily employ liquid cooling.
TrendForce identifies five major components in the current liquid cooling supply chain for GB200 rack systems: cold plates, coolant distribution units (CDUs), manifolds, quick disconnects (QDs), and rear door heat exchangers (RDHx).
The CDU is the critical system responsible for regulating coolant flow to maintain rack temperatures within the designated TDP range, preventing component damage. Vertiv is currently the main CDU supplier for NVIDIA AI solutions, with Chicony, Auras, Delta, and CoolIT undergoing continuous testing.
GB200 shipments expected to reach 60,000 units in 2025, making Blackwell the mainstream platform and accounting for over 80% of NVIDIA's high-end GPUs
In 2025, NVIDIA will target CSPs and enterprise customers with diverse AI server configurations, including the HGX, GB200 Rack, and MGX, with expected shipment ratios of 5:4:1. The HGX platform will seamlessly transition from the existing Hopper platform, enabling CSPs and large enterprise customers to adopt it quickly. The GB200 rack AI server solution will primarily target the hyperscale CSP market. TrendForce predicts NVIDIA will introduce the NVL36 configuration at the end of 2024 to quickly enter the market, with the more complex NVL72 expected to launch in 2025.
TrendForce forecasts that in 2025, GB200 NVL36 shipments will reach 60,000 racks, with Blackwell GPU usage between 2.1 to 2.2 million units.
However, there are several variables in the adoption of the GB200 Rack by end customers. TrendForce points out that the NVL72's power consumption of around 140kW per rack requires sophisticated liquid cooling solutions, making it challenging. Additionally, liquid-cooled rack designs are more suitable for new CSP data centers but involve complex planning processes. CSPs might also avoid being tied to a single supplier’s specifications and opt for HGX or MGX models with x86 CPU architectures, or expand their self-developed ASIC AI server infrastructure for lower costs or specific AI applications.
Share on:
Suggested Items
Bransys Acquires Two REHM VisionXP+ Nitro 3850 Reflow Ovens
10/14/2024 | BransysBransys Group, a leading provider of comprehensive PCB design and assembly services, is pleased to announce the addition of two REHM VisionXP+ Nitro 3850 Type 834 reflow ovens to its state-of-the-art manufacturing facility.
Jabil Acquires Mikros Technologies
10/03/2024 | BUSINESS WIREJabil Inc., a global leader in design, manufacturing, and supply chain solutions, announced the successful acquisition of Mikros Technologies LLC, a leader in the engineering and manufacturing of liquid cooling solutions for thermal management, completed October 1.
NVIDIA Blackwell Platform and ASIC Chip Upgrades to Boost Liquid Cooling Penetration to Over 20% in 2025,
09/23/2024 | TrendForceTrendForce’s latest reports reveal that the launch of NVIDIA’s Blackwell platform, expected in 4Q24, is set to significantly drive the adoption of liquid cooling solutions. Liquid cooling penetration is projected to grow from around 10% in 2024 to over 20% in 2025.
Molex Releases Report on Thermal Management Challenges and Opportunities for I/O Modules
05/22/2024 | PRNewswireMolex, a global electronics leader and connectivity innovator, has published a report that examines thermal management pitfalls and possibilities as data center architects and operators strive to balance high-speed data throughput requirements with the impacts of growing power density and the need for heat dissipation on critical servers and interconnect systems.
The Chemical Connection: Reducing Etch System Water Usage, Part 1
04/05/2024 | Don Ball -- Column: The Chemical ConnectionWater conservation has become an important component of the overall system design for most manufacturing operations today. Changing climate conditions and increasing populations are beginning to strain the freshwater supplies in many areas of the country. As a result, as equipment suppliers, we see an increasing number of requests for options that reduce water usage in proposed wet processing systems. Etching systems tend to use more water because of the need for close temperature control to maintain steady etch rates and the rinsing requirements for complete removal of corrosive etchants from the surface of the product before the next process step. This column contains some simple suggestions for reducing water usage in etch systems that won’t strain the budget too much.