The changing state of the Dynamic Random-Access Memory (DRAM) industry is more than just changing. Demand is through the roof, spurred on by the rapidly growing artificial intelligence (AI) industry, creating new and uncharted challenges. DRAM further is an essential component to data centers powering a large swath of accelerators including Graphics Processing Units (GPUs). These elements are an indispensable backbone for quickly working through massive data sets. In short, monumental growth for NVIDIA is fostering a new boom cycle in demand for DRAM. This surge in demand is pulling supply away from other uses and pushing prices through the roof.
In the past, the industry in this segment of DRAM has traditionally been boom and bust. In 2022, prices crashed as a result of oversupply and a slowing in the pace of data center development. Today, with unprecedented increase of demand, it presents a difficult Catch-22. Experts say that all of these supply issues would take innovation or new fabrication facilities—commonly referred to as fabs—to solve.
Understanding DRAM’s Role in AI
DRAM is the beating heart of today’s computing ecosystem, and more specifically, within today’s AI data centers. It underlies everything for efficiently storing and retrieving data as it powers the GPUs and accelerators that do these massive calculations. The logic tile architecture found in a DRAM tower is complex and interesting. It usually goes about 750 micrometers thick and is made up of multiple stacked memory dice.
The most recent specification for High Bandwidth Memory (HBM4) supports up to 16 DRAM dice stacks. In the last year at least, Samsung has demonstrated its ability to produce a 16-high stack employing hybrid bonding methods. This recent discovery indicates that a 20-die configuration should be achievable before long. This step forward is very important. The DRAM now connects directly to the processor on one side and integrates tightly with the GPU on the other, increasing performance and efficiency at the same time.
With demand reaching a fever pitch, the DRAM industry is at an inflection point. The pattern of major historical overproductions leading to an eventual bust and collapse has often come with greatly damaging impacts. “In general, economists find that prices come down much more slowly and reluctantly than they go up,” stated Mina Kim, highlighting the challenges in stabilizing prices amidst fluctuating demand.
The Investment Landscape in DRAM Manufacturing
To keep up with insatiable demand for DRAM, semiconductor technology leaders are exploring a number of innovative approaches. Two primary approaches exist: enhancing innovation in existing technologies or constructing new fabs to increase production capacity. Each method carries significant financial implications.
Constructing new fabrication facilities is a massive undertaking itself, often costing over $15 billion or more for cutting-edge DRAM fabs. Even leaders in the industry are taking an incremental approach to expanding their capacities, or further increasing expected yields through new package designs made possible with advanced packaging techniques.
Shawn DuBravac remarked on this dual approach, stating, “Relief will come from a combination of incremental capacity expansions by existing DRAM leaders, yield improvements in advanced packaging, and a broader diversification of supply chains.” This strategy would achieve three main objectives: improve domestic production, reduce future risks by addressing supply chain vulnerabilities, and save taxpayer dollars.
Economic Implications and Future Outlook
The economic state of today’s DRAM market is indicative of larger phenomena taking place within the tech sector as a whole. At the very least, NVIDIA’s recent growth has been nothing short of exponential. Their data center business revenue exploded from almost $1 billion in Q4 2019 to a staggering $51 billion by Q4 2025. This rapid growth highlights the never-ending appetite for compute resources powered by cutting-edge memory technologies.
As the market works through these challenges, the specter of the DRAM industry’s cyclic boom and bust history is unavoidable. These recent increases caused by supply limitations are unlikely to go away right away, Kim noted. “DRAM today is unlikely to be an exception to this general observation,” she stated, highlighting the slow recovery process typically seen in such markets.