AMD’s David McAfee On Why AI PCs Are ‘Here To Stay’ And Why ISVs Will Flock To The NPU

In an interview for CRN’s AI PC Week, AMD executive David McAfee talks about why the company believes AI PCs are ‘here to stay’ and why it needed to ship AI PC chips in volume to enable mass-market, AI-enabled computers before many applications are optimized on the chip’s NPU.

While Intel plans to launch its Core Ultra processors for AI PCs next week, rival AMD believes it has a major advantage in the emerging category with more than 50 laptop designs using its first-generation Ryzen AI chips out in the market now and a second wave of AI-enabled chips coming soon.

To AMD executive David McAfee, the fact that the chip designer has already sold millions of x86-based Ryzen AI processors in laptops from big names like HP Inc. and Lenovo is an important differentiator from Intel and Qualcomm, the latter of which makes Arm-based chips for Windows PCs.

“If you look at the volume of AI PCs or AI-enabled PCs that are shipping in the market, I am absolutely certain that we are, by an overwhelming majority, the volume leader of AI-capable PCs today,” said McAfee, corporate vice president and general manager of AMD’s client channel business.

But while Santa Clara, Calif.-based AMD has enabled millions of AI PCs this year with its Ryzen 7040 processors, most of the AI-enabled applications and experiences that are the reason for this category’s existence do not yet take advantage of a key component within AI PC chips from AMD and rivals: the neural processing unit.

Also known as the NPU, the component exists on the die alongside the CPU and the GPU of AMD’s Ryzen AI processors, which the company sometimes refers to as APUs. And it’s designed to offload sustained AI workloads from the CPU and GPU using lower power to improve the laptop’s battery life.

At AMD’s Advancing AI event in San Jose, Calif., Wednesday, the company shed further light on its vision for the AI PC and unveiled the next-generation Ryzen 7040 processors along with a Ryzen AI software toolchain to enable AI-focused ISVs to take advantage of the NPU.

In an interview for CRN’s AI PC Week, McAfee talked about when he expects more ISVs to enable AI experiences on the NPU, why they see the NPU as a compelling component, and how the company’s new Ryzen AI software toolchain will unleash development by “not a few ISVs but millions and millions of enthusiasts who want to build the next killer AI-enabled application.”

“Across the gamut of applications that are focused on data analysis, on productivity, on security and threat detection, the way that AI has the opportunity to be enabled across so many different spaces that are important to knowledge workers I think will be really amazing to watch over the coming months, years, whatever timeline,” he said.

McAfee also discussed why AMD believes AI PCs are “here to stay” and why the company needed to ship chips in volume to enable mass-market, AI-enabled computers before many applications are optimized on the NPU to deliver on the vision shared by the chip designer and its rivals.

“[The 7040 is] one wheel of the bicycle and the other wheel is all of the developer enablement and software tools. I think if you don’t have those two things, you don’t get that traction and you don’t get that exponential growth in applications targeted at this new engine,” he said.

What follows is an edited transcript of CRN’s in-depth interview with McAfee, who also explained why the company’s upcoming Ryzen 8040 processors only feature a boost in NPU performance, why it bet on NPU technology several years ago and how quickly AI PCs could evolve in the future.

Why did AMD put an NPU inside what became the Ryzen 7040 series?

I think that at that point in time, and I’ll probably get the year wrong when it occurred, but it was several years before the product launched. I think the way that we generally plan our product road maps is we think about the in-house organic technology that we have with what’s happening with our CPU cores and our GPU cores and all of those other things. We also spend a lot of time with our ecosystem partners, with Microsoft, with large ISVs to understand where they’re going as well.

And I think that was a moment in time where we realized that the amount of conversation that was beginning around AI and local inference was gaining traction very, very quickly.

And I think that it was one of those things where we saw an opportunity to do something very unique and differentiated with the product. And then, as we worked more and more with our partners, getting the alignment with Microsoft, who enabled [Windows] Studio Effects, etc. as time went on, [we] realized that was really the right bet to place at that point in time.

But it really started with understanding where these kinds of experience makers in the ecosystem were going and how they were thinking about the future. And then consequently, how that translated into the hardware capabilities we needed to build into our chips to help them realize that vision.

You said ‘several years’ ago. The Xilinx acquisition closed in February 2022, [from which AMD got the NPU technology for its Ryzen AI processors]. Does that mean that AMD knew that it wanted to have some kind of NPU but maybe it wasn't what Xilinx had at the time?

I guess what I would say is that when it when it comes to the way that we design products, there’s many sources for where different capabilities come from, both internally organically grown as well as capabilities that we partner with external companies to build for us or with us.

And, yes, I think that when it comes to NPU technology, the timeline for developing silicon is rather long. So I think this is something [where] we were looking at NPU technology from multiple different sources many years ago. And completely unrelated to anything else that was going on, there was just a clear and natural fit with the capabilities that Xilinx had in their IP to how it would integrate into an APU that we could build. It was very fortuitous.

You talked about AMD anticipating use cases. Throughout the planning and development stages of the Ryzen 7040 series, what were the anticipated use cases closer to the beginning and then how did AMD’s views of those anticipated use cases change over time? Because obviously the last couple of years alone have just been pretty crazy [with emerging AI use cases].

I think the anchor of Microsoft’s vision was one: how to use a local inference engine for much more sophisticated audio and video processing. It was clearly something that we wanted to enable. I think the second part of that is when we looked at many of the large ISVs in the ecosystem, there was, at the same time, a lot of experimentation happening around different AI capabilities.

And I think we’ve all seen many of these large ISVs don’t think deployment on an NPU first. They think, ‘Let’s go develop the functionality in the cloud, let’s maybe migrate to a GPU on a device and then when we see the right value in terms of capability or performance or efficiency, that’s the time to migrate to an NPU.’

And so I think that, as time went on and as we got more visibility into what many ISVs were working on in their labs, behind closed doors, I think it became very clear to us that the number of candidates for—I’m not going to say ‘applications’ because it’s really more functions or workflows or things like that—that made the presence of an NPU incredibly interesting to them.

If you break it down, we talked a lot this week about [how] in any APU that we build there’s three different compute elements. You have CPU cores, which, with the right instruction set extensions, have the ability to very efficiently do low-latency, high-throughput inference operations. You have GPU cores, which I think we all know for GPU compute, GPUs are great at highly parallel compute. But as general-purpose compute engines, the overhead that both of those have is that they are very big, very heavyweight engines that require a lot of power to spin them up and you don’t necessarily achieve the efficiency.

I think especially as you look at workloads that have—it’s probably maybe not exactly the right word—but a ‘heavier’ duty cycle, if you want to think about it that way, which would have a more dramatic impact on battery life for a more sustained AI workload. That is the sweet spot for what the NPU is for.

So if you go back to that example of audio or video processing, we’re talking in the time frame where post-COVID, the world’s changed. Videoconferencing is a part of what we all do, day in and day out. Some of us have the pleasure of being on video calls for six-plus hours a day, and you think of how that translates into battery life if you’re spinning up a CPU or GPU core to do all that processing. That experience at a notebook system level would be pretty terrible—unless you’re plugged in.

And so that that example of audio/video processing is kind of the perfect highlight of high-duty cycle when it’s being used, enhanced capability that can’t be achieved without completely sacrificing the user experience and a big efficiency benefit that comes along with bringing that experience onto the NPU.

I wanted to ask about the Ryzen 8040 series. What led AMD to create that product? Because the main change from the 7040 series is the improvement in the NPU performance.

I think that first and foremost, you’re right. The enhancement in NPU performance is the star of the show when it comes to the 8040 over the 7040 series products. I think at its core, the 7040 series and even the 8040 series are very, very capable mobile platforms when it comes to performance or whether it comes to energy efficiency and battery life. We think that we have just an absolute leadership product with the 7040 series.

And the other thing that I would say is, as we got closer to the launch of the 7040 series, as we talked to more and more of our OEM customers, as we talked to more and more software developers and end customers, the demand that we heard more than anything else is it was really centered around, ‘This NPU capability is really interesting, but we would like more capability.’

And one of the interesting things about the architecture that we use for our NPU, the XDNA architecture, is that it is a segmented- or column-based architecture, where there is security separation between having multiple AI workloads running in parallel in that engine. And so we see a future that’s not a distant future, a closer-in future that really will look like, because of this wave of interest and all these new experiences coming, your NPU is not a single task throughput engine anymore, it’s a multitasking machine. And providing the right security and separation around those different workflows running through the NPU is something that we felt was, again, a unique value proposition that we could offer.

So between customer demand, between the uniqueness of our architecture, and then I’ll say just the third leg of it is, in the year that we have lived with the 7040 series, the opportunities that we saw for advancing the state of the art in hardware and software in the way that we define the product and define the silicon, we felt it was absolutely the right thing to do.

So you said that the 100 AI-enabled experiences on the Ryzen 7040 series are enabled mostly by the GPU right now [while a few use the NPU]. Can you talk about why so much of that is concentrated on the GPU side right now? Is that because of the maturity of software development?

I think it’s absolutely maturity of the software ecosystem. I think from an ISV standpoint, it is the first target that many ISVs migrate to when they move from cloud to a local device, and I think it’s a stepping stone to moving more capability and functionality to an NPU once they understand more about how those functions are used, how frequently they’re used and how core they are to the overall workflow.

And when do you expect more experiences to be enabled by the NPU?

I’d say, from large ISVs, it’s a rolling thunder, where every month you’ll have new capabilities brought online. I may be jumping to another question here, but what the heck: I think the other part of it is, our other big announcement this week was the Ryzen AI software toolchain, and I think that is the other thing that has the potential to be really disruptive and transformational for enthusiasts and developers and creators and modders who want to get their hands on, ‘How do I use this engine and do something on my own with it?’ And so I actually think that there’s going to be just a groundswell of excitement and adoption around what that means.

That toolchain works with the [Ryzen] 7040 series and 8040 series, and it allows somebody to go out to Hugging Face, download a model, run it through the AMD toolchain, which quantizes the model, packages it with [the Microsoft-supported] ONNX [runtime], allows, in a matter of minutes with a really simple operation, to deploy that on the NPU. I think that opens the door to just the creativity of not a few ISVs but millions and millions of enthusiasts who want to build the next killer AI-enabled application or students or whatever the case may be.

The AI experiences that you listed for AI PCs were all around content creation. Speaking to CRN’s audience, which is very much commercial client-focused, do you have any visibility into when we will start seeing AI experiences enabled by Ryzen AI that will benefit knowledge workers?

I would say that I probably can’t point to a specific timeline because each ISV has their own development timeline that they’re working on. But what I will say is across the gamut of applications that are focused on data analysis, on productivity, on security and threat detection, the way that AI has the opportunity to be enabled across so many different spaces that are important to knowledge workers I think will be really amazing to watch over the coming months, years, whatever timeline.

And I think that’s in addition to what you heard Microsoft talk about [yesterday at our event] with Copilot being just a more and more integrated part of the PC experience. I think that there’s something that’s really powerful there as well that will give knowledge workers new tools and capabilities that they simply don’t have access to today.

AMD has over 50 laptop designs now with NPUs. Some people are probably thinking, why are AMD, Intel and others putting out these processors with AI engines that are currently not being used much? But I think what you would say is, we need that to happen in order to unleash the development.

I’ll use an example. So rewind to the launch of the first 1000 series Ryzen desktop processors. Up to that point in history, the vast majority of users in the world that were buying desktop processors had available to them at most a four-core processor. And Ryzen came in with eight cores, and honestly, it took time for the software ecosystem to mature and to really adapt to use all of that multi-core processing capability. And now you’ve got 16 cores in desktop. Heck, you’ve got the high-end desktop [chips] that go up to 96 cores. The software ecosystem over a relatively short period of time matured very quickly, learned how to use that hardware capability and delivered better software.

I think it is absolutely true that hardware leads software. And I think that a similar trend will happen with Ryzen AI, where the hardware has got to be there. It’s got to be in the hands of developers. You can’t emulate this. You can’t give somebody something that isn’t real hardware to run it on because at the end of the day, these factors around latency to generate tokens [for AI models] and the overall responsiveness of the experience are critical to whether a software developer believes that this is the killer function to integrate into their application, and you’ve got to have hardware to do it.

So I think, at the root, hardware always lead software, and I think the software ecosystem will respond very, very positively. Again, I go back to having the 7040 in market is one leg of the stool. I don’t how many legs I can describe here. Maybe it’s a two-legged stool. Maybe it’s more of a bicycle. [The 7040 is] one wheel of the bicycle and the other wheel is all of the developer enablement and software tools. I think if you don’t have those two things, you don’t get that traction and you don’t get that exponential growth in applications targeted at this new engine.

I appreciate you explaining that because I think, to me, it's apparent that’s what you need to do, but I think there are probably some people who wonder why [you’re making an NPU-accelerated laptop a mass-market product already.]

Taking a step back, it is new. It is unfamiliar. And I think that as an industry, anything that’s new and different and unfamiliar is always questioned as to what value does it have, why, etc. And then oftentimes, it doesn’t take that much time before it becomes a really valuable capability. And I can think of many examples, but I think this is a perfect one, where a couple of years from now, we will not be talking about at all whether an NPU is a valuable capability in a mobile processor. It will just be taken for granted because the experiences it delivers are just going to be stellar.

In 2024, we will have AMD, Intel, Qualcomm and Apple as PC silicon players. And granted, Qualcomm is starting from a much smaller footprint, but they’re hoping that things go their way in terms of winning more laptop designs and winning more market share in the PC space. What would you say sets AMD’s AI PC strategy apart from those other companies?

I would say Apple almost has to be separated out from that path because the other three are all tied together with Microsoft as the common thread. So there’s a Microsoft-Apple dynamic that will play out that is tangential or orthogonal to whatever might happen in the Windows ecosystem.

I would say within the Windows ecosystem, there’s a couple of things that set AMD apart. I think that first and foremost, we are, of those three, the only CPU or APU provider who is delivering AI at scale today. If you look at the volume of AI PCs or AI-enabled PCs that are shipping in the market, I am absolutely certain that we are, by an overwhelming majority, the volume leader of AI-capable PCs today.

Looking at Qualcomm’s aspirations versus AMD and Intel, there are certain segments in the market that will be very challenging for Qualcomm to really have a foothold and grow. That could be in places like the gaming space, where so much of the content that those users care about is really built for an x86 ecosystem. It potentially translates into the commercial space as well, where legacy applications and the years of building IT infrastructure around x86 also has a lot of inertia that carries it forward.

So I think that AMD having both the scale as well as an architecture that is proven out with hundreds of millions of devices over decades and decades gives us a very unique position to lead from, and I think we absolutely intend to capitalize on that.

HP recently said that the computers it will brand as AI PCs won't arrive until the second half of next year, although the company has also said that new laptops coming out sooner are part of those efforts. At the same time, AMD is talking about AI PCs that are in the market today. Intel is talking about AI PCs it will enable coming out in the next few months. Why are the OEMs, at least in the case of HP, on a different level than Intel and AMD when it comes to defining an AI PC?

I’ll talk about our view of AI PC. On the OEM front, I think what I would say is every OEM is at a different stage in their own AI journey and their own realization of how AI plays into their PC story. And maybe that’s what drives some of the difference.

I think from an AMD standpoint, because we took that first step and enabled AI with [Ryzen] 7040 series and brought an NPU into the x86 space first and delivered initial experiences with Microsoft and continue to grow that ecosystem, the first-generation AI PC is absolutely here. And I think that experience, even for somebody who buys a first-generation AI PC, will continue to get better over time with more applications and experiences that take advantage of that local inferencing processing.

That being said, where does the future go? The AI PC of 2023 is not going to be the AI PC of 2025 or 2027 or pick your year. It still struck me when [AMD CEO] Lisa [Su] said today that it was a year ago that the world first started talking about ChatGPT, and honestly, if you think about that for a second, how quickly the world has changed in that dimension, that’s crazy. That’s absolutely crazy.

And I think that there is a possibility that the same could happen on the device side, where the AI experience that we are able to deliver in this generation versus what that AI experience looks like two or five or seven years down the road will probably, at this moment in time, be completely incomprehensible.

But I think the way the world has transformed in this past year says to me, at least, you can’t rule it out. There could be some very exciting and fundamentally transformative experiences that come out of an AI PC over the years to come. But I think from our standpoint, where we sit today with what we’ve enabled today and what we’re enabling through developers as we speak, the AI PC is here, and we’re leading that wave on x86.

From all of this, I think it’s safe to say that you think and AMD thinks that AI is just going to be kind a crucial aspect of PCs in the future or at least laptops. Is that the case?

If you’re askingi if this a fad that fades away, I think the answer to that is absolutely not. I think one of the things that is maybe a proof point to that is when you have industry behemoths like Microsoft and content creation providers like Adobe and others investing millions or billions of dollars into R&D for new capabilities, building end-user campaigns around awareness of AI capabilities and what they bring to that user experience, I think this is here to stay.

And I think even the fascination that the world has had with things like ChatGPT, as it has evolved, or Stable Diffusion is a prime example of if this can be harnessed and turned into something that really changes the user experience. I have heard people talk about this before: AI has the potential to be as transformational to the PC experience as the mouse was.

And you think about that, for 50 years, keyboard and mouse have been the way that we have interfaced with our PC. But you think about ChatGPT or an AI-based speech-to-text and you think about the way that human-machine interfaces change to be less about you having to make every decision, you doing everything manually, and instead, I can give you natural language commands to my device, who then knows what to do with it and gives me output. That could be game-changing.

And I think that’s the vision that a lot of companies see. In so many ways, from user experience, to productivity, to creation, to everything else, [this] has the opportunity to be truly transformational to the experience, and I think that’s what we all see is the promise of AI.