For most of the last two decades, a data center hall was a story about moving air. Raised floors, hot and cold aisles, containment, computer room air handlers, chillers on the roof. It worked because a well-loaded enterprise rack drew somewhere between 5 and 15 kW, and even dense virtualization rarely pushed past 20 kW. Air could carry that away.

AI accelerators broke that assumption. The heat a modern GPU rack produces is no longer something a stream of cold air can practically remove, and the gap is widening every product generation. That single fact is reshaping how AI facilities are designed, powered, and built.

The number that changed everything

Air cooling has a ceiling, and it is lower than most people expect. In practice, pushing much past 20 to 40 kW per rack with air becomes difficult, loud, and inefficient. You can reach the top of that band with rear-door assistance and careful containment, but the physics gets unforgiving. Air has a low heat capacity, so removing more power means moving more air, which means more fan energy, more noise, and diminishing returns.

Now look at what a current accelerator draws. An NVIDIA H100 has a thermal design power around 700 W. An HGX B200 GPU sits near 1,000 W. AMD’s MI300X lands around 750 W. Put eight of those on a baseboard, add CPUs, memory, networking, and power conversion losses, and a single node is already a serious heat source.

The clearest example is the GB200 NVL72. That is 72 Blackwell GPUs in one rack, drawing roughly 120 kW nominal, with measured figures reported closer to 132 kW under load. There is no realistic amount of air you can blow through a standard rack footprint to carry away 120 kW. Air cooling is not merely inefficient at that density. It is off the table.

What the cooling options actually look like

There is a spectrum of approaches between pure air and full immersion, and each fits a different density band. The table below is the shorthand version our engineers reach for when we scope a build.

Rear-door heat exchangers

The gentlest step beyond air is a rear-door heat exchanger. The rack still breathes air, but a liquid-cooled coil in the back door catches the hot exhaust before it re-enters the room. It buys headroom into the 40 to 60 kW range and slots into an existing air-cooled hall without ripping everything out. It is a bridge, not a destination, for the densest AI racks.

Direct-to-chip liquid

Direct-to-chip, also called cold-plate liquid cooling, is what makes Blackwell-class density possible. A cold plate sits directly on the GPU and CPU packages, and coolant flows through it, taking heat away at the source rather than after it has spread into the air. For a 120 kW rack this is not one option among several. It is effectively mandatory. The hottest components are cooled by liquid, while a smaller residual air or rear-door loop handles the rest of the box.

Immersion

Immersion cooling submerges whole boards in a dielectric fluid. It reaches the best efficiency numbers and the highest densities, and it removes fans almost entirely. The trade is a very different serviceability and facility model, which is why it tends to appear in specialist and edge builds rather than mainstream enterprise AI halls today.

Efficiency is the quiet win

Density gets the headlines, but efficiency is where liquid pays back. Power usage effectiveness, or PUE, is the ratio of total facility power to the power that actually reaches the IT gear. A PUE of 1.0 would be perfect. Air-cooled halls commonly run between 1.4 and 1.6, meaning 40 to 60 percent overhead on top of the compute itself, much of it spent spinning fans and chillers.

Liquid changes the math. Direct-to-chip designs routinely land under 1.2, and immersion can push under 1.1. At AI scale that difference is enormous. On a multi-megawatt cluster, moving from 1.5 to 1.15 can free up a megawatt or more of capacity that was previously going to cooling overhead. That is capacity you can either sell, redeploy to compute, or simply not have to build.

The part nobody can bolt on later

Here is where liquid cooling stops being a rack decision and becomes a building decision. Once coolant is running to the chip, the facility has to be designed around it, and most of that design cannot be retrofitted cheaply.

• Coolant distribution units. CDUs sit between the facility water loop and the technical loop that touches the servers. They manage flow, temperature, and pressure, and isolate the clean secondary loop from the building supply. They need space, power, and redundancy planned in from the start.
• Secondary loops and manifolds. Every rack needs supply and return piping, quick-disconnect couplings, and manifolds sized for its flow. That plumbing is part of the row layout, not an afterthought.
• Floor loading. Filled piping, CDUs, and 120 kW racks are heavy. Structural loading has to be checked against the real, fully populated weight, not a nominal figure.
• Leak detection and containment. Water and electronics demand real detection, drip trays, and automatic isolation. This is a design discipline, not a sensor you add at the end.
• Heat rejection. All that captured heat still has to leave the building, through dry coolers, cooling towers, or increasingly heat reuse. The outdoor plant has to match the indoor density.

None of this survives being treated as an add-on. Pipe routing, structural capacity, power distribution, and heat rejection are interlocking constraints. Get the sequence wrong and you are cutting concrete and re-running loops after the fact, at many times the cost. This is exactly why liquid-cooled AI has to be planned with the building, which is the core of how we approach infrastructure building: power, cooling, structure, and cabling designed as one system.

The takeaway

Liquid cooling is not a niche preference for the most aggressive operators anymore. The moment your roadmap includes Blackwell-class accelerators, direct-to-chip liquid moves from optional to structural. It unlocks the density these clusters need, it cuts the efficiency overhead that eats into usable capacity, and it forces cooling into the earliest conversations about the building itself. The organizations that treat it that way, as a facility discipline rather than a rack accessory, are the ones whose GPU cluster builds come online on time and stay efficient as density keeps climbing.