IDTechEx Forecasts Demand for HBM to Grow 15-fold by 2035 for HPC and AI
Author: Dr Shababa Selim, Senior Technology Analyst at IDTechEx
Parallel computer architectures, such as those deployed for modern high-performance computing (HPC) and artificial intelligence (AI) workloads, frequently encounter memory bottlenecks that limit performance, known as the 'memory wall'. Historically, this has stemmed from the advances in processor performance (e.g. CPUs, GPUs, and other AI accelerators) outpacing developments in memory performance. This poses significant challenges for data-intensive HPC and AI workloads, where bottlenecks lead to under-utilization of processors. IDTechEx's report “Hardware for HPC, Data Centers, and AI 2025-2035: Technologies, Markets, Forecasts” critically assesses the key developments and trends in memory and storage technologies, including high-bandwidth memory (HBM), DDR, NAND, and many more.
Memory bottlenecks for HPC and AI are driving the segmentation of memory hierarchy with near-memory HBM, memory expansions, and high-performance storage drives. Source: IDTechEx
Memory bottlenecks are driving segmentation of memory hierarchy
Traditionally, the memory hierarchy consists of CPU/GPU cache (e.g., SRAM), main memory (e.g., DRAM), and storage drives (e.g., SSD, HDD). However, HPC and AI workloads are driving a segmentation of this hierarchy due to memory wall issues and underutilization of processors. This is fueling developments in in-package HBM, memory expansions through CXL-pooled memory, NAND SLC based storage memory solutions, and high-performance data storage solutions such as TLC and QLC NAND.
However, it is worth noting that further segmentation of memory hierarchy introduces layers that may also hinder high-speed operation due to increased data movement between layers.
HBM: A key enabler of parallel processing HPC and AI
HPC uses parallelization to break down problems across multiple nodes, requiring the fast transfer of large data volumes. HBM's high bandwidth allows it to handle multiple memory requests simultaneously from various cores, making it essential for GPUs and accelerators. It is no surprise then that the overwhelming majority of GPUs and accelerators use HBM to facilitate parallel workload processing. Additionally, continued development is pivotal to improving accelerator performance, with HBM4 offering a higher capacity and improved performance expected to launch in 2025.
HBM is essentially a 3D structure of vertically stacked DRAM dies on top of a logic die and relies on advanced packaging technologies such as through silicon vias (TSVs) and uses silicon interposer for interconnection with the processor. The current generation HBM3E uses thermal compression with micro-bumps and underfills to stack DRAM dies however, manufacturers such as SK Hynix, Samsung, and Micron are transitioning towards more advanced packaging technologies, such as copper-copper hybrid bonding for HBM4 and beyond to increase input/outputs, lower power consumption, improve heat dissipation, reduce electrode dimensions, and more.
Unit sales of HBM are forecast to increase 15-fold by 2035, compared to 2024, according to IDTechEx. Source: IDTechEx
Impact of geopolitics and outlook
Geopolitical tensions and US-China trade tensions are having a notable impact on the memory landscape. For instance, in September 2022, the US began restricting the export of high-end GPUs to China, with the ban expanding over time. In response, Chinese firms like Huawei and Baidu have been stockpiling HBM modules from Samsung for AI applications according to a Reuters report. There's also a huge investment in China's semiconductor industry. Additionally, China is heavily investing in its semiconductor industry. While SK Hynix, Samsung, and Micron currently lead the HBM market, Chinese companies like Huawei, CXMT, and Tongfu Microelectronics are emerging, filing patents related to HBM and some even entering early-stage production. Amid a booming HPC and AI hardware industry, IDTechEx forecasts a 15-fold increase in HBM unit sales for HPC by 2035, compared to 2024 levels.
For a detailed analysis of HPC and AI technologies and market outlooks, see IDTechEx's report ” Hardware for HPC, Data Centers, and AI 2025-2035: Technologies, Markets, Forecasts.” This report offers a 10-year forecast covering the overall HPC hardware market and key hardware technologies, including CPUs, GPUs, memory HBM, storage, server boards, and thermal management. It also covers networking and interconnects in HPC and the advanced semiconductor packaging technologies that enable HPC.
For more information on this report, including downloadable sample pages, please visit www.IDTechEx.com/HPC.
Further information on advanced semiconductor packaging technologies for HBM and other HPC applications can be found in the IDTechEx report “Advanced Semiconductor Packaging 2025-2035: Forecasts, Technologies, Applications”.
For the full portfolio of semiconductors, computing and AI market research, please visit www.IDTechEx.com/Research/Semiconductors.
