AWS CEO ‘Bullish’ On Homemade Chip Future: ‘We Can Optimize Like Crazy’

“People enjoy a narrative where they say like, ‘How are you possibly doing your own chips when you have other partners who have their other chips?’ And it turns out, customers like choice,” said AWS CEO Matt Garman.

Amazon Web Services CEO Matt Garman is bullish about his company’s future in homemade silicon chips as AWS plans to “optimize like crazy” in the semiconductor industry with its unique ability to “control” the entire process thanks to its massive infrastructure footprint.

Owning their own chipset is helping AWS customers in various ways—from better performance to lowering their carbon footprint, such as with its new Graviton4 processor.

“Graviton4 absolutely outperforms the best other x86 processors at a 20 percent lower price. So many of our customers get 40 percent to 50 percent price performance gains while also using less power and improving their carbon footprint using Graviton,” Garman said who first joined AWS in 2006. “And it’s because we control that whole process.”

“We don’t have to build these processors to run in a general purpose environment. They’re going to run exactly in our server, exactly in our data center, exactly with our networking stack and so we can optimize that just for our customers,” said Garman. “We can optimize like crazy around that, plus we have a very good team that’s building the chips.”

Garman said AWS’ initial decision to build custom chips in-house was to support its virtualization technology via Nitro and ability to dictate customer cost. “That means that we don’t have to go buy those from third parties, which allows us to lower our cost,” said Garman.

‘AWS Is The Best Place To Run’ Nvidia, Intel And AMD Processors

The Seattle-based cloud giant has invested over a decade in designing custom silicon optimized for cloud computing and storage-intensive workloads. About five years ago, Garman said AWS turned its attention to creating AI processors such as AWS Inferentia machine learning chips and Trainium ML processors for training deep leaning models.

“We think that there are some use cases where our own custom processors can help customers save money,” said Garman.

However, Garman points out that AWS is partnering with the world’s most innovative and largest chipmakers like Intel, AMD and Nvidia.

“We firmly believe that AWS is the absolute best place to run Intel, to run AMD, to run Nvidia processors and we think that we can offer some differentiated capabilities by offering our own processors as well,” AWS CEO said.

Garman highlighted Nvidia, in particular, as an all-star processor supplier that AWS has built a tight partnership with. “We’re partnering super closely to build a giant AI infrastructure for them to build their own models and to run their own test cases inside of AWS,” he said. “Because they realize that we have the best operating environment and the best performance in order to run their own servers. So we have a great partnership together.”

‘Not All Workloads Will Work Better On’ AWS

Garman made clear that AWS will continue to invest both in its own silicon chip innovation as well as close go-to-market and integration partnerships with other market leaders.

“Not all workloads will work better on our own processors, but we feel very bullish about the opportunity there. Trainium is the newest chip that we have out, which is very focused on large-scale training clusters for these AI models,” said Garman.

AWS plans to launch its new Trainium2 processors by the end of this year.

“We feel incredibly excited about that platform. We think that we have the opportunity to aggressively lower cost for customers while increasing performance,” said Garman. “There’s going to be a breadth of processor options for customers for a long time, and we think more choice is better for our customers.”