The rapid evolution of the DRAM (Dynamic Random-Access Memory) industry today is underpinned by a rare perfect storm of circumstances. A recent historic boom and bust cycle in memory production colliding with an unprecedented demand for artificial intelligence (AI) hardware. Skyrocketing prices for DRAM has sprung from this perfect storm. More importantly, it is the unsung hero of enabling GPUs and other accelerators that we now find in every DC.
With AI technologies maturing every day, the demand for powerful, fast memory solutions has reached an all-time high. Now DRAM comparatively sits at the tip of the spear, with critical support for generative AI data processing and workloads. The newly-found profitability that comes with DRAM deployments for these applications has added more pressure on supply, driving prices to all-time-high levels. According to industry experts, conditions are about to turn a corner on shortages. They say this progress will be driven by new production methods and greater manufacturing capacity.
Current Supply Dynamics
The memory market has seen huge turmoil in the recent past. In 2022, prices for DRAM, the most frequent sort of memory, included a steep drop thanks to a normal oversupply. In order to react to this, manufacturers cut production in half. This strategy was meant to prop up prices which had dropped below the cost of producing sugar. That new AI-driven demand has further muddled the market.
High-Bandwidth Memory (HBM), a newer, specialized and advanced variant of DRAM, has proven especially critical. HBM chips are designed to include several vertically stacked DRAM dies. The upcoming HBM4 standard will allow for 16 stacked dies, beating the current maximum of 12. This new stacking capacity is key to keeping pace with burgeoning demand for data-intensive applications.
Small scale DRAM chips are particularly important to data centers. Today, almost 2,000 new facilities are in the planning or construction pipeline worldwide. Each individual DRAM tower has a thickness of only 750 micrometers. It connects to a dozen of GPUs and other accelerators through up to 2,048 micrometer-scale connections. The dizzying complexity of such structures only goes to further accentuate the deep and pernicious ties between memory and processing power that is quickly defining modern computing environments.
“There are two ways to address supply issues with DRAM: with innovation or with building more fabs.” – Mina Kim
Future Production Plans
Meanwhile, looking forward, the three Korean companies dominant in the DRAM industry are investing a massive $230 billion to expand production capacity substantially. Samsung, for example, has told investors that it expects to start producing DRAM from a new plant in Pyeongtaek, South Korea, by 2028. This expansion is expected to be a key driver in mitigating some of the supply pressures that are currently impacting the market.
Production innovations are on the horizon. Samsung showed that it can provide a 16-high stack of DRAM with hybrid bonding. They even proposed that a 20-die stack might be within reach in a near future. This new breakthrough is sure to have a great impact on expanding memory and meeting the needs of the ever-growing field of AI technologies.
Shawn DuBravac, an industry analyst, notes that relief from supply constraints will stem from several factors: “Relief will come from a combination of incremental capacity expansions by existing DRAM leaders, yield improvements in [advanced packaging], and a broader diversification of supply chains.”
The Economic Perspective
With smart, sustainable production expansions on the horizon and new innovations around slate, the future is looking bright. Economic fundamentals tell us to be more wary of price stabilization. Mina Kim points out that “in general, economists find that prices come down much more slowly and reluctantly than they go up.” This particular observation is especially true for DRAM, even more so considering the persistent high demand for computational power.
The complexities of supply and demand movements in the DRAM market are indicative of larger economic trends. Manufacturers are adjusting quickly to change at both the technological level and consumer level. Consequently, equilibrium might take longer to achieve than many anticipate.
“In general, economists find that prices come down much more slowly and reluctantly than they go up. DRAM today is unlikely to be an exception to this general observation, especially given the insatiable demand for compute.” – Mina Kim