In the AI gold rush, chip-makers are selling shovels – and they are in short supply. As the latest generation of artificial intelligence models like ChatGPT look set to transform our lives, the hardware that makes it all possible is becoming a strategic asset, with countries, companies and researchers all scrabbling to get hold of supplies. With talk of shortages lasting until at least next year, some would-be AI developers face being left behind.
At the centre of this is Nvidia, which joined the earlier this year, sitting alongside the likes of Apple and Google owner Alphabet. Nvidia, based in California, made its name creating hardware to run computer games, but these graphics processing units (GPUs) turned out to be extremely capable at churning through the vast amounts of data required to train an AI. Reports suggest that Nvidia now controls .
But there are signs that this huge demand is and creating a bottleneck for those wanting to get their hands on Nvidia’s most powerful AI GPUs, the H100 and A100. This is a predictable result of the current AI boom, with recent advances being closely linked to scale and therefore the number of GPUs you have.
Advertisement
The company hasn’t confirmed a shortage and declined an interview with żěè¶ĚĘÓƵ, but any new orders are unlikely to be fulfilled until 2024 – a huge stretch of time when it comes to the pace of AI development.
“NVIDIA allegedly has sold out its whole supply through the year,” Suhail Doshi, founder of AI image creator Playground AI, on 25 July in response to conversations he said he has had with vendors. “I’d forecast a minimum spend of $10m+ to play right now if you can even get GPUs.”
On top of this seeming shortage, AI teams in China are feeling the bite of US restrictions that came into force in September last year. The US has banned exports of high-end GPUs, like the H100, to Russia and China in order to .
Nvidia has attempted to skirt this export ban by developing versions of the chips with reduced data transfer rates that comply with the ban, and is seeing healthy sales. TikTok owner ByteDance has so far this year, perhaps aiming to get ahead of any . ByteDance didn’t respond to a request for comment.
Meanwhile, other Chinese firms are suffering. Tech giant Alibaba recently told investors it had been unable to meet customer demand for training AIs due to a shortage of GPUs.
Worst hit by the global GPU shortage is academia, given its limited research budgets. “The hype around machine learning has led to a lot of competition for kit and brains,” says Tobias Weinzierl at Durham University, UK. “Even if we order [GPUs], we don’t get kit in a timely fashion.” Meanwhile, higher salaries and better access to resources are tempting many academic researchers to leave for the private sector, says Mhairi Aitken at the Alan Turing Institute in London.
at Liverpool John Moores University, UK, says that industry is taking AI ideas, architectures and research from academia and applying massive scale – which university budgets can’t stretch to – to leverage better results. What is needed, he says, is more collaboration – to bring together the experience and talent of academia with the budgets and experience of commercialisation from big tech companies.
“Companies look for money, they don’t care about human development. As academics, we work hard to serve human beings,” says Al-Jumeily. “Whatever AI has reached now, it’s because of academia. Without our research, they cannot push those boundaries.”
, also at Durham University, says that tech companies working on exceedingly large data-driven AI models, equipped with vast amounts of hardware, are leaving academic researchers behind.
“Companies can exploit extremely large corpora of data [that] we will never be able to host or handle given our limited storage capabilities,” he says. “On top of that, large corporations now actively involved in applied and more theoretical research have stormed into the international conference and scientific journal landscape, making it very hard for small academic institutions to compete.”
at chip-maker AMD says that academics can apply for grants to get GPUs, or buy time on cloud-based services, but concedes that the costs can rise quickly. He says he is seeing a trend for some researchers to focus on smaller AI models that are trained to do specific tasks, leaving research on the gigantic models that require computing hardware worth millions to the big technology companies. “That’s a lot more economical,” he says.
Fine-tuned model
The other approach academics are taking is to focus on a different part of the process of training an AI model. The biggest AI systems are built from scratch using huge datasets to create software that has broad abilities. Then a fine-tuning stage occurs to tweak the model and hone its abilities on a single task. This stage uses a smaller but more focused dataset and needs fewer computing resources.
“For researchers that may not have many GPU resources, they can still do many things in the fine-tuning stage and they can try to improve this work or improve this field,” says  at Nanyang Technological University in Singapore.
One issue with this approach is that the researchers often don’t have full details of how the model was trained and what data was used, making it harder to carry out open and repeatable science.
Can anything be done to increase the supply of chips for AI? Numerous companies are racing to launch their own GPUs to rival Nvidia, while tech giants like Google, Amazon, Microsoft and Meta are turning to custom chips to run their own AI tools as efficiently as possible.
But switching chips isn’t easy. Nvidia’s GPUs utilise proprietary software called CUDA that squeezes the most out of them when applied to AI-related tasks, and this isn’t available on chips made by the likes of Intel and AMD.
George Hotz, the founder of AI company Comma, said in a recent video posted to YouTube that he is “” after encountering problems trying to write and run custom AI software on the company’s GPUs. James Knight at the University of Sussex, UK, says that Hotz’s complaint was a “version of the experience everyone who has tried to use AMD hardware has suffered”.
Balasubramanian says it is true that some users have found problems making the switch from Nvidia to AMD because the platforms are slightly different. This can be solved, he says, if developers invest more time learning the new environment.
“People who have gotten into the CUDA ecosystem early, for them it is this system of familiarity. And the expectation is everyone matches that familiarity,” says Balasubramanian. “It requires a little extra tweaking because, at the end of the day, the architecture for Nvidia and AMD are not the same.”
That said, there are software tools available to help make switching between chips a smoother experience. If researchers build their AI using software such as PyTorch (originally developed by Meta) or TensorFlow (originally created by Google) they can theoretically switch from Nvidia GPUs to AMD GPUs with minimal upheaval. This means they can change their supplier based on performance, cost or supply chain issues – a benefit also enjoyed by the likes of Meta and Google.
“That’s what these big AI players want,” says Balasubramanian. “And that’s part of the reason why they developed this – they want to make sure that they’re not locked into a single [GPU] vendor.”
There are other positive signs, too. Supply chain problems during the covid-19 pandemic are resolving, the race to buy hardware to capitalise on a new trend is slowing down and the number of GPU manufacturers is expanding. New entrants are working to catch up and the market is widening, while existing players step up production.
Ultimately, if advanced AI models are going to be fully integrated into society, we need a wide range of people, from academics to activists, to be able to explore and interrogate them. Only then will the wealth of the AI gold rush be shared by all, rather than be limited to a few firms.
