AWS CEO Matt Garman: 'World will benefit from choice' in AI chips — Amazon Web Services CEO Matt Garman

Making more choice available to customers for their AI computing needs is at the core of Amazon Web Services’ chip procurement strategy, which includes designing its own hardware and acquiring high-end chips from market leader Nvidia.

This is according to AWS CEO Matt Garman who, in response to a question from TechCentral on Tuesday, said the cloud services giant’s two-pronged chip strategy is less about mitigating against supply-chain risks – this is despite high demand for advanced chips and limited manufacturing capacity worldwide – and more about ensuring AWS customers can access a variety of hardware options best suited to their AI computing needs.

“At the foundry level, both chips are made at the same foundries, so I don’t know [if our strategy] mitigates anything. I don’t think the processor world is a winner-take-all environment, so it is less about supply-chain risk here and more about giving customers the most choice,” said Garman, who spoke at a media event held by AWS at its Seattle, US headquarters on Tuesday, which TechCentral attended. “We invest in both because we think the world will benefit from choice.”

Amazon’s journey to designing its own chips began with the acquisition of Israeli microelectronics firm Annapurna Labs in 2015. Annapurna’s design capabilities led to the deployment of AWS’s first AI chips, Inferentia, in 2019. Inferentia was followed by first-generation Trainium chips in 2022. These chips, which run the bulk of AI workloads at AWS data centres, are now in their second generation, with a third-generation chip in the works.

Garman’s assertion that AWS’s two-pronged chip strategy is not centred on mitigating against supply-chain risks is rooted in the fact that both the AWS Trainium and Nvidia H100 GPU’s are manufactured by Taiwan’s TSMC, meaning any bottlenecks at TSMC could affect the supply of both chips.

TSMC is one of only two companies in the world – the other is South Korea’s Samsung Electronics – with the ability to manufacture chips using the 4- or 5-nanometre technology in the H100 or Trainium2 chips. The upcoming Trainium3 chips will be made using an even more advanced 3nm process.

Cheaper to build and run

Although not as powerful, Trainium chips are cheaper to build and run compared to market-leading Nvidia H100 GPUs. AWS claims training AI models on Trainium2 chips is 30-50% cheaper than on Nvidia’s H100s. Even so, Amazon procures plenty of Nvidia chips for its data centres.

Garman said different customer workloads call for different hardware requirements. At the highest level, this philosophy is evidenced by the different hardware deals AWS has struck with AI companies.

The first, with Claude developer Anthropic, involves a multibillion-dollar investment by AWS into Anthropic. Anthropic, in turn, has named AWS as its primary infrastructure provider, leading to the bulk of workloads for training and using Claude models running on AWS Trainium hardware.

Read: Absa turns to AWS in expanded cloud banking push

AWS has struck another deal with ChatGPT creator OpenAI. Announced on Monday, the deal involves a $38-billion commitment by OpenAI to buy compute capacity from AWS over the next seven years. OpenAI’s workloads in the AWS cloud will run on Nvidia hardware and not Trainium chips.

These moves come as AI investments ramp up across the market, with an anticipated surge in agentic AI adoption at the enterprise level expected to fuel a surge in AI computing demand.

“There is a $125-billion investment in capex for the AWS business, and it’s growing. Part of that is because we have massive demand, requiring more computers, data centres and power. That demands a massive investment from us, and we have to make that investment well ahead of demand,” said Garman.

Another likely driver spurring AWS’s diversified chip strategy is competition. Both Google and Microsoft, which offer Google Cloud and Microsoft Azure cloud services, respectively, follow similar strategies to AWS regarding the deployment of AI chips in their data centres. For all three “hyperscalers”, Nvidia chips represent the peak of performance, while partnerships with other chips manufacturers such as AMD for Google and Broadcom for Microsoft allow for the provision of a second, more cost-effective offering.

“We are really excited about Trainium; with both chips we have now and the road map going forward, we think we can deliver a lot of value for customers. The Nvidia team is an incredible executor. They have awesome processors, they have been leading this whole revolution for the last few years, and I don’t expect that to change, either.