Scaling Liquid Cooling for AI: What You Need to Know

Today's large language models generate heat at a rate that would have seemed absurd to data center engineers a decade ago. A single AI accelerator rack can now draw 60–100 kW or more — compared to the 8–12 kW that conventional enterprise racks averaged not long ago. Air cooling, the industry's default for fifty years, is hitting a physical wall. The fans get bigger, the cold aisles get colder, and at some point, you're spending more energy moving air than you are running compute. Something had to give.

Liquid cooling has become the answer the industry is circling. But "liquid cooling" is not a single technology, and deploying it at scale inside real facilities — with real budgets, real legacy infrastructure, and real clients who have opinions — is considerably more complicated than the vendor pitch decks suggest.

What Liquid Cooling Actually Is (And Why It's Different)

At its core, liquid cooling moves thermal energy using water or a dielectric fluid instead of air. That sounds simple. The implementation is not.

The two dominant approaches are direct liquid cooling (DLC), where coolant runs through cold plates mounted directly on processors, and immersion cooling, where servers are submerged entirely in a non-conductive fluid bath. Each has a different risk profile, infrastructure footprint, and maintenance behavior. A third approach — rear-door heat exchangers — acts more as a hybrid, capturing heat from air before it escapes the rack. Organizations often choose incorrectly because they're matching a solution to a brochure rather than to their actual load profile.

The efficiency advantage is real and measurable. Traditional air-cooled data centers typically operate at a Power Usage Effectiveness (PUE) of 1.4–1.6, meaning 40–60 cents of every energy dollar goes to overhead rather than compute. Well-implemented liquid cooling systems can push PUE toward 1.1 or below. At hyperscale, that delta translates to tens of millions of dollars annually and a materially smaller carbon footprint — which matters increasingly as utilities, regulators, and corporate sustainability commitments tighten the screws on data center operators.

The Deployment Challenges Nobody Talks About Enough

Here's where the conversation gets honest. Liquid cooling works. Getting it deployed, however, exposes a set of organizational and technical friction points that catch facilities teams off guard.

The first problem is the building itself. Most existing data centers were not designed for the structural loads, leak detection requirements, or piping infrastructure that liquid cooling demands. Retrofitting a colocation facility or enterprise data center means routing coolant distribution units (CDUs) through spaces that were never meant to accommodate them, often without disrupting live production environments. That is a real constraint — not a theoretical one.

The second problem is the IT-facilities divide. Liquid cooling requires coordination between the teams that manage servers and the teams that manage buildings. In many organizations, those teams barely speak. The network engineers ordering next-generation GPU clusters and the facilities managers responsible for mechanical systems are often making decisions on different timescales, with different budgets, and with different risk tolerances. Rajat Bhagat of Arcadis, one of the engineering firms working at the intersection of these disciplines, has flagged this integration gap as one of the primary deployment bottlenecks — and it's underappreciated.

Fluid management is its own discipline. Leak detection, water quality monitoring, corrosion inhibitors, glycol concentration — these aren't IT problems, but they're now living inside IT infrastructure. Organizations that treat liquid cooling as a drop-in upgrade rather than a new operational paradigm tend to find out they were wrong at the worst possible moment.

Client adoption is also slower than the headline numbers suggest. While hyperscalers like Google, Microsoft, and Meta have moved aggressively into liquid cooling, the mid-market enterprise is still largely watching. The capital cost of transition, combined with uncertainty about which specific AI workloads will dominate their environments in 24 months, creates rational hesitation. Nobody wants to build the wrong system.

Why AI Workloads Make Liquid Cooling Non-Negotiable

Artificial intelligence isn't just another workload. It's a thermal event.

NVIDIA's H100 GPU — the chip that's become the de facto standard for large-scale AI training — has a thermal design power (TDP) of 700 watts per chip. Dense configurations pack dozens of these into a single rack. The H200 pushes further. Whatever comes next will push further still. Air cooling at these densities isn't just inefficient — it's architecturally impossible without compromising compute density to a degree that defeats the purpose of the hardware.

This is where liquid cooling stops being a preference and becomes load-bearing infrastructure. For AI data centers specifically, the integration challenge runs deeper than thermal management. AI training jobs are highly sensitive to interconnect latency — meaning GPU placement is constrained by networking topology, not just thermal zones. Liquid cooling infrastructure has to be designed around the compute layout, not retrofitted around it afterward. Getting the sequencing wrong adds cost, limits density, and can introduce performance penalties that show up directly in training times.

There's also an energy source consideration that's gaining traction. As data centers come under pressure to operate on cleaner power, liquid cooling's efficiency advantage compounds. A facility running at PUE 1.1 rather than 1.5 needs meaningfully less total power capacity — which, in a constrained grid environment, can be the difference between getting a project permitted and not. Clean energy technology and liquid cooling infrastructure are not separate conversations anymore.

Future-Proofing: The Hardest Question in the Room

The honest answer to "how do I future-proof a liquid cooling deployment?" is that you can't — not completely. And anyone who tells you otherwise is selling something.

What you can do is build in flexibility margins that reduce the cost of adaptation. IT refresh cycles in AI infrastructure are compressing. The window between major GPU generations is now roughly 18–24 months. Each new generation brings higher TDP, different form factors, and potentially different cooling interface requirements. A facility designed specifically around H100 cold plate specs may need retrofit work when the next architecture arrives.

The smarter approach is designing to a thermal envelope rather than a specific chip. If your CDUs, piping, and rack infrastructure are specified for 120 kW per rack, you have headroom to absorb the next generation — and possibly the one after that — without a facility-level overhaul. This requires capital discipline upfront, because you're building capacity you're not using yet. But the alternative — tight-fitting infrastructure that has to be ripped and replaced every cycle — is far more expensive over a ten-year horizon.

Scalability planning also means thinking carefully about water availability, especially in data center markets like Phoenix, Las Vegas, or parts of Texas where water stress is already a policy concern. Closed-loop systems and facilities designed to minimize water consumption aren't just environmentally responsible — they're an increasingly practical hedge against future regulatory exposure.

Long-term cost modeling should include stranded asset risk. If you sign a 15-year lease on a facility and build cooling infrastructure optimized for 2024-era AI hardware, you're making a bet. Know what assumptions that bet depends on, and stress-test them before breaking ground.

What Comes Next

The firms that will execute liquid cooling deployments well over the next five years are the ones treating it as an integrated infrastructure discipline rather than a mechanical add-on. That means engineering partners who understand both the IT layer and the building systems layer, clients willing to invest in operational training, and procurement processes that account for the full lifecycle — not just the installation day cost.

The infrastructure challenges are real, but they're solvable. The technology is mature enough to deploy at scale. The economic case, particularly for AI data centers where compute density is non-negotiable, is increasingly clear. What's still catching up is the organizational capability to execute: the project management, the cross-functional coordination, and the willingness to treat liquid cooling as a new operational domain rather than a conventional HVAC upgrade with a fancy name.

The facilities that get this right in the next 24 months will have a structural cost and performance advantage over those that delay. In an environment where AI compute capacity is a competitive asset, that advantage compounds quickly.

Explore more about liquid cooling solutions in our marketplace!

[INTERNAL LINK: liquid cooling technologies]

[INTERNAL LINK: AI data center efficiency]

[INTERNAL LINK: infrastructure challenges in data centers]