Lenovo’s Vlad Rozanovich: ‘There’s Definitely A Run On H100-Type GPUs’

In a recent interview with CRN, Vlad Rozanovich, Lenovo’s top server sales executive, says partners should consider alternatives to Nvdia’s most powerful and supply-contrained data center GPUs such as the H100 to power their customers’ AI workloads. ‘Understand the customer’s use case and you will find and create business opportunities,’ he says.

Lenovo is adding to the chorus of companies that are seeing a shortage of Nvidia’s most powerful data center GPUs due to high demand driven by generative AI development, and the company’s top server sales executive said channel partners should consider other options if possible.

The Chinese IT giant reported in its first-quarter earnings report last month that GPU supply constraints prevented Lenovo from delivering more AI systems, which contributed to an 8 percent decline in the company’s Infrastructure Solutions Group revenue.

[Related: Lenovo: 'Partners Can Make A Lot Of Money With Us' On AI Compute And Storage]

In a recent interview with CRN, Vlad Rozanovich, the head of Lenovo’s worldwide infrastructure sales, said the supply constraints are around powerful Nvidia GPUs such as the H100 and A100. The H100 in particular is designed to handle massive, so-called Transformer models that include many large language models at the center of generative AI applications today.

“No question. There’s definitely a run on large language model-, H100-type GPUs,” said Rozanovich, who was Lenovo’s North America president until taking the new infrastructure job in June.

While Rozanovich said this is an issue for organizations that want to train the largest of the large language models, he is trying to fight the perception among Lenovo’s channel partners and customers that everyone needs H100s or A100s for their AI applications.

“It’s what people think is the impression because [those are] the one[s] with the longest lead times. They’re like, ‘Oh my goodness, how am I going to get my hands on an A100 and H100?’ The reality is, understand the customer’s use case and you will find and create business opportunities,” he said.

How Lenovo Is Helping Partners Find The Right AI Hardware

Rozanovich said there are plenty of readily available GPUs, like Nvidia’s new L40S universal accelerator, that are “much more cost effective” in areas such as computer vision for fast food and retail store systems. He added that organizations “can even do AI on CPUs” if they’re looking for systems to run inference on smaller AI models for live applications.

“I’ve actually been talking to channel partners about this over the last couple of weeks. When you’re engaging on GPU-related sales, don’t think that you have to just sell the top-end large language modeling-based systems,” he said.

Instead, partners should work closely with customers in understanding their requirements to devise the best system configurations for their use cases, which, in many cases, may not require H100s or A100s, according to Rozanovich, who worked at Nvidia rival AMD for 24 years before joining Lenovo.

“The reality is, depending on the customer use case, the software application stack and what you’re doing, that’s the opportunity for channel partners to figure out and actually coach customers to say, ‘You don’t need what everybody’s buzzing about,’” he said.

“There actually is availability on the GPU side, whether it’s these other products like [Nvidia’s] A40, L40S [and] L4. There’s this other set of products that actually is available that may be a better solution set for that particular customer use case,” he added.

To help North American partners navigate the complexities of configuring AI systems, Lenovo is enlisting data center staff from the continental group to educate the channel on “what is that right stack, what is that right solution set and what is the right portfolio,” according to Rozanovich.

“Because it’s hard to do it where you just put up a selling card that says, ‘OK, if you do this, then this, if you do this, then this.’ It’s a complicated thing based on the customer vertical and the type of solution and the application that’s running,” he said.

“The thing that we would encourage is if you’re a channel partner and you want to engage with customers around AI, make sure that you’re going back to our channel teams and our end-user teams to talk about what some of those customer solution sets could be,” Rozanovich added.

Partner Says Customers Have Various Options For AI Workloads

From the view of an executive at one Lenovo channel partner, Andy Lin, Nvidia’s shortages have been more acute with the A100, which debuted in 2020, than the H100 that launched last year.

But Lin, CTO at Houston-based Mark III Systems, said these shortages are not an issue for most AI applications because they don’t need processors that are as powerful as the A100 or H100.

“It’s sort of the 80/20 rule, where 80 percent of jobs are small, but you have many, many users. 80 or 90 percent of users probably don’t need to run these large Transformer jobs,” he said.

“However, the 20 or 10 percent that do, they’re the ones that take up all the resources, because they’re the ones that are building each model,” Lin added.

The executive agreed with Rozanovich that Nvidia’s new L40S is a “great versatile card” that can handle plenty of AI and graphics workloads well and said customers could also consider mixing up what kind of processors they use for training and inference.

“Maybe you look at different ways to break up your pipeline, so that you offload certain parts of the AI pipeline onto different GPUs that are specialized for certain areas,” Lin said.

Another route customers should consider in determining the best way to power AI applications is by using software to maximize utilization of existing processors, according to the executive. One vendor he name-checked is Run:AI, which lets organizations split up GPUs into smaller instances.

“They do a good job of taking existing GPUs of any generation, not just the latest, and actually allowing you to basically partition them for different users,” Lin said.

Nvidia Plans To Ramp Up Supply Over Next Several Months

Nvidia recently promised that it plans to ramp up supply for its most in-demand GPUs and associated hardware over the next several months.

Addressing supply constraints in its second-quarter earnings call, Nvidia CFO Colette Kress said the company has “developed and qualified additional capacities and suppliers for key steps in the manufacturing process, such as CoWoS packaging.”

“Our supply over the next several quarters will continue to ramp as we lower cycle time and work with our supply partners to add capacity,” she said.

Kress added that Nvidia’s new L40S GPU “will help address the growing demand for many types of workloads from cloud to enterprise.”