With AI Chips Getting Hotter, Liquid Cooling Vendors See Big Channel Play

AI chips are running much hotter than their predecessors, giving rise to a new class of channel-friendly startups focused on liquid cooling for high-power CPUs and GPUs.

Udi Paret says he’s been humbled by the “very fast, high-magnitude changes” that his liquid cooling company, ZutaCore, has experienced over the past several months.

The main driver for these changes is the same thing that has caused tectonic shifts elsewhere in the tech industry: the frenzied development of generative AI offerings and the way many of these solutions require powerful accelerator chips like Nvidia GPUs, which are becoming increasingly hotter due to growing power requirements.

Paret, ZutaCore’s chief business officer, knew that the intensifying temperatures of these AI chips in a growing number of cases could not be chilled by traditional air cooling and would instead require liquid cooling technologies like the ones developed by ZutaCore, a direct-to-chip liquid cooling vendor with headquarters in San Jose, Calif.

[RELATED: CRN’s 2024 AI Special Issue]

But he couldn’t have anticipated the extent to which liquid cooling would become “an acute need” so quickly for new AI data centers being built, with customers now starting to call the shots on how they want to cool the chips inside.

“They are now demanding liquid cooling and prescribing which liquid cooling and then, together with the manufacturers like us, are going to the server OEMs and ODMs and are shopping around which one can now deliver the platform,” he said.

While sever vendors like Hewlett Packard Enterprise and Supermicro have been developing their own liquid cooling technologies, pure-play vendors like ZutaCore say there is not only room for companies like them to grow in the space but ample opportunity for solution providers to partner with such companies and share in the rewards of a new growth engine.

A recent report by research firm Dell’Oro Group said the data center liquid cooling market is expected to reach more than $15 billion over the next five years.

“[Solution providers] are important because to fulfill the customers’ configurations at scale, many of them are using both multinationals as well as a number of regional players depending on the customer situations. And we are certainly investing a lot in nurturing these relationships,” Paret said.

The ZutaCore executive said liquid cooling technologies can help solution providers grow their business in part because they’re becoming an “integral part” of the set of hardware and software technologies needed to operate modern data centers.

But Paret said the channel also stands to make a lot of money reselling, integrating and supporting such technologies because there is so much demand right now.

“It’s still a premium market, which means premiums both on the hardware but also the integration services [are] going to be simply higher,” he said.

Why New Chips Are Creating A Greater Need For Liquid Cooling

One liquid cooling vendor that has built its business from the ground up with the channel in mind is Accelsius, an Austin, Texas-based company that is commercializing direct-to-chip cooling technology initially developed by Nokia Bell Labs.

“If you think about value-added integrators, rack-and-stack integrators, typically data center operators have been using these partners for a long time because they have deep relationships and they’re trusted, so to not go through that channel seems a little bit absurd to us,” said Josh Claman, a former Dell Technologies sales executive who is now CEO of Accelsius.

Like some other pure-play providers, Accelsius is focused on developing liquid cooling technologies that are meant for high-performance CPUs and GPUs.

Claman said his company’s direct-to-chip technology is meant to last multiple generations of chips because companies like Nvidia, AMD and Intel have set expectations that new releases will likely come with big leaps in power requirements.

Case in point: Nvidia said the most powerful version of its next-generation Blackwell GPU set to debut later this year will require 1,200 watts, a 71 percent increase from its H100 that was released in 2022. Consequently, this model will require liquid cooling, which the chip designer plans to integrate into its own rack-scale systems for the GPUs.

These increases in power consumption are also happening with CPUs. Intel, for instance, said its upcoming Xeon 6900P series will scale up to 500 watts, a nearly 30 percent increase from the previous generation that debuted in late 2023.

“As these chips go from 200 to 400 to 1,000 to 2,000 watts per socket, you don’t want to be called back as a reseller and [have the customer say], ‘You sold us this solution three years ago and it can already not cool my latest server chips,’” Claman said.

Like leaders of other liquid cooling providers, Claman doesn’t think the opportunities end with AI. But he does see AI as the “penetration point” for solution providers to sell offerings to customers for traditional data centers, which could allow them to lower carbon emissions, reduce energy allocated to cooling and, as a result, save on operating costs.

“After they get used to liquid cooling and realize there’s just a huge economic savings, then I think it’s going to be widely adopted,” the executive said.

How Liquid Cooling Can Help Solution Providers Sell More Servers

When Bernie Malouin invented the liquid cooling technology that would serve as the basis for his company, JetCool Technologies, he didn’t realize how much an innovation like his, which was being used to cool things like radar and airplane equipment, could play a key role in the rise of AI computing several years later.

“I had no idea that today I would be sitting here with AI having come into the industry with devices where 1,000 watts is almost becoming commonplace,” said Malouin, who is CEO of the Littleton, Mass.-based company.

With liquid cooling, Malouin sees an opportunity for solution providers to increase the number of servers they sell to customers for AI data centers because of how such technologies can help end users overcome power budgets.

That’s because JetCool’s liquid cooling technologies can use less energy than traditional air-cooling technologies in data centers, according to the CEO.

“When we’re selling either our full liquid cooling solutions or even our starter kit through Dell, which still reduces the electricity consumption 15 percent, what that means is that the customer, for every seven servers, they can get an eighth one in for free in that power budget,” Malouin said. “That’s good for the customer. They get more compute in that rack. It’s also good for the channel partners because they sell more servers into that rack.”

Edge Computing To Also Benefit From Liquid Cooling

While liquid cooling will be necessary for the build-out of many new AI data centers, Nathan Blom, a former Lenovo executive who is now co-CEO of Sheffield, U.K.-based liquid cooling vendor Iceotope, said there will also be significant opportunities for solution providers to sell and integrate such technologies at the edge.

That’s because edge computing locations often deal with significant power restraints since operators need to site them primarily based on how close they can get to users for collecting and serving data, according to Blom.

“If I’m using 40 percent of my power budget for an air handler in a hut, that’s very wasteful. If I can get that 40 percent back [by switching to liquid cooling] and use it for compute, that makes the site much more valuable,” he said.

The edge is also where a great deal of AI inference is expected to happen because of the need to keep latencies as low as possible, according to Blom. And that means there’s even greater impetus for edge computing sites to implement liquid cooling technologies.

“The traffic is going to drive through the edge, just the network traffic alone, so you’ve got to push some of that inferencing out of the data center to the edge,” he said.

‘Imperative’ For The Channel To Understand Liquid Cooling Landscape

With liquid cooling set to become pervasive in data centers in the coming years, Blom thinks that it’s “imperative” for solution providers to understand the evolving landscape and how competing technologies differ from each other.

“I would advise them to start getting smart on this and training their sales staff to speak to it,” he said.

It’s also important for the channel to listen to what kind of solutions customers are looking for, according to the Iceotope leader, because “OEMs are struggling a little bit to know where to place their bets right now.”

“There are a lot of options out there as to what to invest in, and they’re looking for customer feedback as to how to design their next n-plus-one, n-plus-two generation servers that will eventually flow down to the channel partners,” Blom said.