Servers, Racks And RAG: Dell Technologies’ Five Big AI Advances

Dell Technologies now says it offers AI data readiness with Dell Data Lakehouse, the densest GPU-capacity inside liquid-cooled racks and an AI model-buffing Agentic RAG with Nvidia.

Dell Technologies says its AI advances in servers offer customers the densest GPU capacity available for Nvidia’s most sophisticated processors running in liquid-cooled cabinets that, all together, push AI and data to the limits of what’s possible.

“This is something that we're very, very proud of, and the innovation and the speed of innovation across infrastructure over the last year and a half to two years has been – really as a longtime Dell employee – mind boggling and very, very exciting,” said Dell’s Varrun Chhabra, senior vice president of ISG and Telecom.

Speaking during a media briefing ahead of Microsoft Ignite and SC 24, Chhabra said advances in AI hardware are arriving in the form of bigger compute capacity and more sophisticated orchestration between devices and software to improve inferencing, he said.

But while 80 percent of enterprises have generative AI efforts underway or plan to in two years, many are not ready, whether due to data preparation or a talent gap, Chhabra said. Dell has sought to counter those bottlenecks with turnkey AI products that offer support wrapped around validated designs.

At the SC24 event last month Dell unveiled new servers that pack 96 of Nvidia’s most advanced GB200 processors into a scalable, water cooled, 19-inch wide rack that can be preconfigured and arrive ready to turn on, said Arunkumar Narayanan, Dell’s senior vice president, compute and networking portfolio management, during the media briefing.

“We will integrate all of this from the factory to provide a turnkey solution. So all the customer needs to do is get the rack delivered, plug it in, the liquid will be set up, the air-cooled servers will be set up, and the entire thing will work as one solution,” he said.

During its most recent third quarter, Dell shipped $2.9 billion in AI servers resulting in an AI server backlog of $4.5 billion, while Dell’s five-quarter pipeline grew more than 50 percent sequentially, with growth across all customer types, the company said during an earnings call.

Chhabra said that while many of Dell’s most advanced AI offers are available for channel partners, some “custom capabilities” unique to Dell are not.

“On the services side, and as well with some offers that include integrated professional services such as Integrated Rack Scalable Systems, there are select servers that are available for channel resell,” he told reporters on the call. “The short answer is many of the services we talked about today are available for channel resell. But certain services that involve custom capabilities are not yet available for channel resale.”

Here are five AI advances that Dell has announced:

Dell PowerEdge XE7740

This 4U PowerEdge is promising to deliver enhanced AI productivity with Intel Xeon 6 processors from the “Granite” family of chips. It offers up to 8 Intel Gaudi 3, or 8 Nvidia H200 NVL GPUs, or 16 of Nvidia’s L4 PCIe GPUs.

“This is purpose-built for inferencing and model fine-tuning, especially in the enterprise,” said Dell’s Arunkumar Narayanan. “This is probably the best platform for enterprise adoption to start. Customers have a wide choice of AI accelerators they can put in the box. It starts with as many as eight, double-wide 600-watt GPUs. Or they can get down to 16 single-wide GPUs all the way down to 75 watts. So depending on the workload, depending on the use case, the customer can select different types of choices.”

This is the first PowerEdge server with Intel Xeon 6 processors. It comes with twice the PCIe GPU’s capacity compared to Dell’s previous generation to provide high speed networking to the GPUs, Narayanan said.

The XE7740 is the follow-up to the “incredibly successful” Dell R760. It delivers twice the GPU capacity, as well as additional networking cards

Dell PowerEdge XE9685L

Dell is taking the “L” and applying it to its latest generation of flagship servers with the PowerEdge XE9685L, which stands for liquid cooling. Dell said the 9685L’s Direct Liquid Cooling manifolds offer best-in-class cooling efficiency.

The 9685L can house 24 of Nvidia’s H200 GPUs or Nvidia’s GB200 GPUs inside its 4U chassis, for up to 96 when configured as a rack -- three times better GPU density per rack than Dell’s air-cooled PowerEdge XE9680.

“The key point about this platform is that we can do 96 GPUs in a rack. That will be industry-leading density of GPUs in a rack,” Narayanan said. “It will have 20 percent better connectivity because we added two more PCIe slots and now we have 12 PCIe slots on the server, and we built iDRAC capabilities into the server to help with remote management and leak detection.”

The machine runs AMD’s Turin CPUs at 192 cores per system for the highest CPU core density of any platform, Narayanan said. Dell hopes to deliver the systems out of the factory as a full rack, inside its recently announced rack system.

Dell Integrated Rack 5000 (IR5000)

Built with cloud service providers and large enterprises in mind, the IR5000, integrated rack scalable system, can accommodate both liquid-cooled and air-cooled servers.

Based on a traditional 19-inch standard, the form accommodates the PowerEdge 9680L and the PowerEdge 9685L for up to 96 GPUs per rack. It can also house the PowerEdge XE7740, XE7745 and the best-selling XE9680.

“We will integrate all of this from the factory to provide a turnkey solution so all the customer needs to do is get the rack delivered, plug it in,” Narayanan said. “The liquid will be set up. The air-cooled servers will be set up, and the entire thing will work as one solution.”

Dell Data Lakehouse With Apache Spark

Dell’s single query platform for enterprise data in AI workloads now has Apache Spark as a fully embedded component, Chhabra said. He said it gives organizations the ability to process large-scale data for AI workloads, simplifying analysis.

“All of this is going to reduce the amount of time it takes for AI deployment,” Chhabra said.

That leads to three to five times faster queries and up to a 53 percent reduction in the cost of data analytics.

“This will empower organizations to process large-scale data for AI workloads, helping organizations simplify management, processing and analysis, all within the same cohesive environment that the Dell Data Lakehouse provides,” Chhabra said.

Dell Data Lakehouse has unified access control between the two open-source SQL engines it uses Apache Trino, and Spark, formerly called PrestoSQL.

“This essentially is a dynamic orchestration layer that allows customers to scale efficiently while maintaining the performance that they need for AI workloads,” Chhabra said.

Dell Agentic RAG Framework With Nvidia

This is built on Dell hardware and run with Nvidia software to allow customers to accelerate retrieval augmented generation operations when performing complex queries. The design uses Dell PowerEdge servers, Dell PowerScale storage combined with Nvidia’s NeMo Retriever microservices and the Nvidia AI Blueprint for muiltimodal PDF data extraction to retrieve contextual insights from company data with automated workflows that provide mutli-modal outputs including graphs, images, and tables.

“RAG has transformed the AI adoption landscape,” Chhabra said. “What RAG allows customers to do is to take their proprietary data, and feed it into existing models without a lot of pre-legwork and seamlessly integrate their enterprise data into workflows so end-users have the latest and greatest information about their business to be able to make the best decisions for the business.”

Chhabra said RAG has received a lot of attention since the start of the year, however organizations faced challenges integrating large-scale data for RAG into AI operations. Dell Agentic RAG with Nvidia takes on the challenge with RAG agents working in the background, he said.

“We’ve collaborated with Nvidia to have an innovative approach … this uses RAG agents working in the background to tackle issues such as data fragmentation, compliance challenges, as well as the inability of end-users to create complicated prompts to get answers,” he said.