The artificial intelligence boom is running into a hardware hurdle that industry experts describe as a 'structural bottleneck': memory bandwidth. As AI models grow exponentially in size, the speed at which data can be shuttled between processors and memory is failing to keep pace, threatening to throttle performance and inflate costs for companies racing to deploy generative AI.
Memory bandwidth — the rate at which data can be read from or written to memory — has historically improved more slowly than raw compute power. Today's leading AI chips, such as Nvidia's H100 and the upcoming B200, require vast amounts of high-bandwidth memory (HBM) to feed their processing cores. Yet supply constraints on HBM, manufactured by SK Hynix, Samsung and Micron, have created a logjam. Industry analysts estimate that memory now accounts for up to 40% of the total cost of an AI accelerator, compared with roughly 20% for conventional data centre chips.
For UK investors, the implications are twofold. The FTSE 100 has limited direct exposure to pure-play AI chipmakers, but the London Stock Exchange hosts several companies in the semiconductor supply chain, including IQE and SMT. Pension funds with global equity allocations have significant holdings in Nvidia and AMD. Recent volatility in those stocks — Nvidia shares fell 5% last week on reports of delayed HBM deliveries — has rattled portfolios. 'Memory is becoming the new compute bottleneck,' said Dr. Helena Cross, a semiconductor analyst at London-based CrossTech Research. 'Investors need to watch not just who makes the chips, but who controls the memory stack.'
The bottleneck also carries implications for UK businesses adopting AI. Smaller firms may face longer wait times and higher costs for cloud-based AI services as data centre operators scramble to secure HBM supplies. Meanwhile, energy consumption per training run is rising because processors must idle while waiting for data, adding to electricity bills and carbon footprints. 'This isn't a temporary glitch,' Cross added. 'It's a fundamental physics problem that will shape the next decade of AI hardware design.'
Some in the industry are exploring alternatives, such as near-memory computing and optical interconnects, but these technologies remain years from commercial deployment. For now, the memory bottleneck is a stark reminder that AI's progress depends as much on humble memory chips as on the flashy processors that grab headlines.