The memory chip market is in the midst of historic branded downturns, which is expected to continue for a few years to come. On the other hand, Samsung has been truly impressive. By 2024, they expect to roll out a 16-high stack with hybrid bonding and are confident a 20-die configuration is possible. The current specification for High Bandwidth Memory (HBM4) only permits up to 16 stacked DRAM dies. Most chips in production today, though, use just 12 dies. This gap demonstrates the critical need for creativity and innovation within the industry.
The size of demand for DRAM is absolutely incredible. It pulls supply from other memory use cases and pushes prices to record highs. The historic boom and bust cycle in the DRAM industry has officially hit a tipping point. Compounding this, the demand for expanding AI hardware infrastructure is greatly increasing the demand for chips. Companies are having a hard time growing their capacities to meet demand. Even with this motivation, the prohibitive costs of new fabrication plants, typically over $15 billion, prevent most firms from acting.
Industry Dynamics and Economic Impacts
Yet the COVID-19 pandemic has panic in chip supply that has echoed across the industry. The resulting recession, extended through 2023, has dealt a severe blow. Because of the oversupply issues, major memory and storage companies — Samsung among them — have cut production by up to 50 percent. This drop in output has added to the current backlog of shortages and increased competition between tech companies fighting for their share of these finite resources.
Here, trade expert Thomas Coughlin underscores a key issue. He goes on to describe how companies are only financially able to grow when the economy is booming. When downturns happen, the industry’s research and development of new technologies comes to a standstill. This sets off a vicious cycle, leading to longer-term shortages when demand does eventually come back.
“There are two ways to address supply issues with DRAM: with innovation or with building more fabs.” – Mina Kim
Herein lies one of the industry’s biggest fears. Without serious commitment to innovation or capacity increases, we’re doomed to a perpetual cycle of shortages.
Technological Innovations and Future Capacities
Considering all these difficulties, Samsung’s progress in memory technology could be the game changer we need. If the company can match its ability to make a 16-high stack with hybrid bonding, that should provide some comfort down the road. While they’ve proposed even larger models as possible, which is leading to more efficient processing power and memory scaling.
The DRAM tower as it stands today boasts a thickness of only 750 micrometers. This groundbreaking approach allows us to connect HBMs on both sides of processors. This allows us to combine GPUs and memory together in a single, incredibly powerful unit. Such innovations would open up dramatic new levels of performance, efficiency, and capabilities of the computing systems that underpin every key industry.
Shawn DuBravac, IC Insights’ IT executive, gives an optimistic picture on DRAM future supply situation. He believes that relief will come from not only incremental capacity expansions by existing DRAM leaders but yield improvements in advanced packaging techniques and a broader diversification of supply chains.
“Relief will come from a combination of incremental capacity expansions by existing DRAM leaders, yield improvements in [advanced packaging], and a broader diversification of supply chains.” – Shawn DuBravac
These innovations will be critical to stabilizing the market and serving the insatiable demand for DRAM.
The Road Ahead: Challenges and Opportunities
Despite technological advancements and strategic planning efforts, industry experts caution that relief may not arrive until well into the latter part of this decade. As Intel CEO Lip-Bu Tan recently cautioned, we’re not going to see real improvements in supply for at least 2028.
“There’s no relief until 2028.” – Intel CEO Lip-Bu Tan
This already bleak outlook is made even worse by almost 2,000 new data centers now in the pipeline or being built. The overwhelming pace of this data infrastructure development serves as a testament to our rapacious appetite for compute power and memory capacity. This increase in demand makes the strain on our supply chain even worse.
Mina Kim provides a sobering view on what we can expect to see in price movement across the DRAM market. She notes that historical patterns suggest prices decline much more slowly than they rise, particularly given the current high demand for computing capabilities.
“In general, economists find that prices come down much more slowly and reluctantly than they go up. DRAM today is unlikely to be an exception to this general observation, especially given the insatiable demand for compute.” – Mina Kim