Historically, information facilities have relied on air cooling — the place mechanical chillers flow into chilled air to soak up warmth from servers, serving to them keep optimum situations. However as AI fashions improve in measurement, and the usage of AI reasoning fashions rises, sustaining these optimum situations will not be solely getting tougher and dearer — however extra energy-intensive.
Whereas information facilities as soon as operated at 20 kW per rack, immediately’s hyperscale amenities can assist over 135 kW per rack, making it an order of magnitude tougher to dissipate the warmth generated by high-density racks. To maintain AI servers working at peak efficiency, a brand new method is required for effectivity and scalability.
One key answer is liquid cooling — by decreasing dependence on chillers and enabling extra environment friendly warmth rejection, liquid cooling is driving the subsequent technology of high-performance, energy-efficient AI infrastructure.
The NVIDIA GB200 NVL72 and the NVIDIA GB300 NVL72 are rack-scale, liquid-cooled methods designed to deal with the demanding duties of trillion-parameter giant language mannequin inference. Their structure can also be particularly optimized for test-time scaling accuracy and efficiency, making it a really perfect alternative for working AI reasoning fashions whereas effectively managing power prices and warmth.
Driving Unprecedented Water Effectivity and Value Financial savings in AI Knowledge Facilities
Traditionally, cooling alone has accounted for as much as 40% of a knowledge middle’s electrical energy consumption, making it one of the crucial vital areas the place effectivity enhancements can drive down each operational bills and power calls for.
Liquid cooling helps mitigate prices and power use by capturing warmth straight on the supply. As an alternative of counting on air as an middleman, direct-to-chip liquid cooling transfers warmth in a know-how cooling system loop. That warmth is then cycled via a coolant distribution unit by way of liquid-to-liquid warmth exchanger, and finally transferred to a facility cooling loop. Due to the upper effectivity of this warmth switch, information facilities and AI factories can function successfully with hotter water temperatures — decreasing or eliminating the necessity for mechanical chillers in a variety of climates.
The NVIDIA GB200 NVL72 rack-scale, liquid-cooled system, constructed on the NVIDIA Blackwell platform, presents distinctive efficiency whereas balancing power prices and warmth. It packs unprecedented compute density into every server rack, delivering 40x greater income potential, 30x greater throughput, 25x extra power effectivity and 300x extra water effectivity than conventional air-cooled architectures. Newer NVIDIA GB300 NVL72 methods constructed on the Blackwell Extremely platform boast a 50x greater income potential and 35x greater throughput with 30x extra power effectivity.
Knowledge facilities spend an estimated $1.9-2.8M per megawatt (MW) per 12 months, which quantities to just about $500,000 spent yearly on cooling-related power and water prices. By deploying the liquid-cooled GB200 NVL72 system, hyperscale information facilities and AI factories can obtain as much as 25x value financial savings, resulting in over $4 million {dollars} in annual financial savings for a 50 MW hyperscale information middle.
For information middle and AI manufacturing facility operators, this implies decrease operational prices, enhanced power effectivity metrics and a future-proof infrastructure that scales AI workloads effectively — with out the unsustainable water footprint of legacy cooling strategies.
Transferring Warmth Outdoors the Knowledge Heart
As compute density rises and AI workloads drive unprecedented thermal masses, information facilities and AI factories should rethink how they take away warmth from their infrastructure. The normal strategies of warmth rejection that supported predictable CPU-based scaling are now not ample on their very own. Right now, there are a number of choices for shifting warmth exterior the ability, however 4 main classes dominate present and rising deployments.
Key Cooling Strategies in a Altering Panorama
- Mechanical Chillers: Mechanical chillers use a vapor compression cycle to chill water, which is then circulated via the information middle to soak up warmth. These methods are usually air-cooled or water-cooled, with the latter usually paired with cooling towers to reject warmth. Whereas chillers are dependable and efficient throughout various climates, they’re additionally extremely energy-intensive. In AI-scale amenities the place energy consumption and sustainability are high priorities, reliance on chillers can considerably influence each operational prices and carbon footprint.
- Evaporative Cooling: Evaporative cooling makes use of the evaporation of water to soak up and take away warmth. This may be achieved via direct or oblique methods, or hybrid designs. These methods are way more energy-efficient than chillers however include excessive water consumption. In giant amenities, they’ll devour hundreds of thousands of gallons of water per megawatt yearly. Their efficiency can also be climate-dependent, making them much less efficient in humid or water-restricted areas.
- Dry Coolers: Dry coolers take away warmth by transferring it from a closed liquid loop to the ambient air utilizing giant finned coils, very similar to an automotive radiator. These methods don’t depend on water and are perfect for amenities aiming to scale back water utilization or function in dry climates. Nonetheless, their effectiveness relies upon closely on the temperature of the encompassing air. In hotter environments, they could wrestle to maintain up with high-density cooling calls for except paired with liquid-cooled IT methods that may tolerate greater working temperatures.
- Pumped Refrigerant Methods: Pumped refrigerant methods use liquid refrigerants to maneuver warmth from the information middle to out of doors warmth exchangers. In contrast to chillers, these methods don’t depend on giant compressors inside the ability and so they function with out the usage of water. This technique presents a thermodynamically environment friendly, compact and scalable answer that works particularly nicely for edge deployments and water-constrained environments. Correct refrigerant dealing with and monitoring are required, however the advantages in energy and water financial savings are vital.
Every of those strategies presents completely different benefits relying on elements like local weather, rack density, facility design and sustainability objectives. As liquid cooling turns into extra frequent and servers are designed to function with hotter water, the door opens to extra environment friendly and environmentally pleasant cooling methods — decreasing each power and water use whereas enabling greater compute efficiency.
Optimizing Knowledge Facilities for AI Infrastructure
As AI workloads develop exponentially, operators are reimagining information middle design with infrastructure constructed particularly for high-performance AI and power effectivity. Whether or not they’re remodeling their total setup into devoted AI factories or upgrading modular parts, optimizing inference efficiency is essential for managing prices and operational effectivity.
To get the very best efficiency, excessive compute capability GPUs aren’t sufficient — they want to have the ability to talk with one another at lightning pace.
NVIDIA NVLink boosts communication, enabling GPUs to function as a large, tightly built-in processing unit for max efficiency with a full-rack energy density of 120 kW. This tight, high-speed communication is essential for immediately’s AI duties, the place each second saved on transferring information can imply extra tokens per second and extra environment friendly AI fashions.
Conventional air cooling struggles at these energy ranges. To maintain up, information middle air would have to be both cooled to below-freezing temperatures or stream at near-gale speeds to hold the warmth away, making it more and more impractical to chill dense racks with air alone.
At almost 1,000x the density of air, liquid cooling excels at carrying warmth away because of its superior warmth capacitance and thermal conductivity. By effectively transferring warmth away from high-performance GPUs, liquid cooling reduces reliance on energy-intensive and noisy cooling followers, permitting extra energy to be allotted to computation quite than cooling overhead.
Liquid Cooling in Motion
Innovators throughout the business are leveraging liquid cooling to slash power prices, enhance density and drive AI effectivity:
Cloud service suppliers are additionally adopting cutting-edge cooling and energy improvements. Subsequent-generation AWS information facilities, that includes collectively developed liquid cooling options, improve compute energy by 12% whereas decreasing power consumption by as much as 46% — all whereas sustaining water effectivity.
Cooling the AI Infrastructure of the Future
As AI continues to push the boundaries of computational scale, improvements in cooling shall be important to assembly the thermal administration challenges of the post-Moore’s regulation period.
NVIDIA is main this transformation via initiatives just like the COOLERCHIPS program, a U.S. Division of Power-backed effort to develop modular information facilities with next-generation cooling methods which are projected to scale back prices by not less than 5% and enhance effectivity by 20% over conventional air-cooled designs.
Trying forward, information facilities should evolve not solely to assist AI’s rising calls for however achieve this sustainably — maximizing power and water effectivity whereas minimizing environmental influence. By embracing high-density architectures and superior liquid cooling, the business is paving the way in which for a extra environment friendly AI-powered future.
Study extra about breakthrough options for information middle power and water effectivity introduced at NVIDIA GTC 2025 and uncover how accelerated computing is driving a extra environment friendly future with NVIDIA Blackwell.
