Designing the modern AI cooling stack

In the largest year-over-year jump in AFCOM’s decade of research, the research group’s newest 2026 State of the Data Centre Report has found that the average rack density has climbed from 16 kW to 27 kW in just one year,

Furthermore, the study concluded that 72% of operators expect AI workloads to increase data centre capacity requirements whilst more than 60% of organisations are either already using liquid cooling or plan to adopt it within the next two years.

Scaling direct-to-chip liquid cooling demands a complete rethink of the cooling stack, with the industry subsequently challenged to design integrated, scalable architectures that support next‑generation high‑density compute.

Six paths to heat rejection

In order to select the most appropriate liquid cooling architecture, data centre operators must weigh four primary factors: existing cooling infrastructure compatibility, deployment size, speed of deployment, and energy efficiency.

Three heat rejection methods combined with one of two Coolant Distribution Unit (CDU) form factors create the six most common liquid cooling architectures. Each has advantages and disadvantages, which must be weighed depending on the type of deployment.

The first step is to identify the heat rejection method, which determines how heat is eventually transferred to the outdoors and dictates the type of heat exchange (Liquid-to-Air or Liquid-to-Liquid) used by the system.

1. Reject to Air (Liquid‑to‑Air, existing system): CDU expels heat into data centre air, working with existing air‑cooled infrastructure. Best for small deployments (single server to few racks), where speed matters most.
2. Reject to Facility Water (Liquid‑to‑Liquid, existing system): CDU connects to chilled or condenser water loops. Suited to mid‑/large‑scale sites with chiller plants, prioritising efficiency over rapid rollout.
3. Reject to Independent Water (Liquid‑to‑Liquid, dedicated system): Builds a new heat rejection system for liquid cooling. Ideal for major deployments focused on efficiency and heat reuse, where longer timelines and new piping are feasible.

The next step is then to determine the CDU capacity and form factor. Independent of the heat rejection method, the CDU form factor dictates how cooling fluid is distributed among the IT equipment.

4. Rack‑Mounted CDU: Installed in the rack, creating an isolated TCS loop. Best for 1–10 racks needing rapid install. Limits failures to one rack but becomes less efficient and costlier per kW as scale grows.

5. Floor‑Mounted CDU: Placed on the facility floor, serving multiple racks/rows via a shared loop. Suited for >10 racks where workloads can tolerate common‑cause failure. Offers better efficiency, saves rack space, and lowers cost per kW at scale, but a single failure affects all connected racks.

Retrofitting existing facilities

Integrating high-density AI clusters into existing “brownfield” facilities presents a unique set of engineering constraints. Today, most legacy halls were designed for power densities and cooling methodologies that are fundamentally challenged by the requirements of modern AI accelerators.

Successful retrofits depend on localised heat rejection strategies that minimise disruption to the central:

• Rear-Door Heat Exchangers: Systems stabilise rack temperatures even under peak AI workloads, making them an ideal server cooling option for GPUs.
• Liquid-to-Air Exchangers: For facilities where water access is limited, heat dissipation units (HDUs) can remove heat from AI servers and reject it directly into the room air, eliminating the need to route facility water to the rack.
• Localised Scaling: Direct-to-chip liquid cooling at the rack level allows for staged upgrades, preventing major central plant overhauls and making the modernization of smaller, constrained sites more feasible.

The logic of efficiency – the 7+3 Framework

Sustainability in the AI era is governed by the trade-offs between energy and water consumption. Optimising Power Usage Effectiveness (PUE) requires a sophisticated understanding of approach temperatures (the temperature difference between the primary and secondary cooling loops).

The sustainability and efficiency of a direct-to-chip liquid cooling system are driven by a holistic view of the architecture, from the chip to the chiller.

There are seven core design factors and three operational practices that dictate energy and water consumption:

Seven design factors:

1. Outdoor Heat Rejection: Air‑cooled chillers with economizers save water versus towers, which consume heavily year‑round.
2. IT Inlet Fluid Temps: Raising supply temps (e.g., 25°C → 45°C) cuts compressor energy 30–40% and water use 40–60%.
3. Rack Power Density: Higher AI‑driven densities boost liquid heat capture, slightly improving plant efficiency.
4. CDU Type: Liquid‑to‑Liquid CDUs are more efficient than Liquid‑to‑Air, avoiding extra heat exchange cycles.
5. Component Selection: Efficient pumps/heat exchangers with low approach temps enable higher facility water temps, more economiser hours, and less compressor energy.
6. Coolant Type: PG25 is standard for reliability; deionised water offers better thermal capacity but requires complex chemical management.
7. Heat Re‑use: Higher liquid cooling temps enable waste‑heat recovery for district heating/industry, improving Energy Reuse Factor.

Three operational Practices:

1. Control Scheme: Advanced systems use digital twins and dynamic pump control to match real‑time loads, avoiding wasted energy.
2. Maintenance: Regular coolant monitoring and filter cleaning prevent damage and reduce pump strain.
3. Adaptive Deployment: Modular, phased builds align infrastructure with evolving IT demand, avoiding stranded capacity and excess carbon.

Designing for AI is an exercise in mastering fluid dynamics and material science at scale. Moving away from anecdotal risks toward a physics-based architectural strategy will enable facility engineers can deploy direct-to-chip cooling with confidence.

//By Canninah Dladla, Cluster President for Sub-Saharan Africa at Schneider Electric