Perhaps few chip industries are more challenged than that of memory chips. Soaring demand driven in large part by advances in artificial intelligence and the overall growth of data centers compounds the stress. Large language models, which demand terabytes per second of data, are putting a huge strain on current memory solutions. This article examines the current state of memory production. It further sheds light on the innovations being developed to address persistent supply challenges and clarifies the anticipated timeline for relief from the scarcity.
In 2023, Samsung made a dramatic move, slashing memory and storage component production in half. This brilliant strategic maneuver was designed to prevent artificially low market prices from falling below a producers’ average variable long-run cost of production. In AI, companies are scrambling to catch up with demand from massive data centers and new AI applications. This rollback, in addition to their unique greenwashing gimmick, illustrates an unusual prudence – and admission of failure, perhaps?
Innovation on the Horizon
To fulfill their 2024 milestone, they leveraged groundbreaking hybrid bonding techniques to achieve a successful 16-high stack. This innovative approach enables High-Bandwidth Memory (HBM) to be mounted on both sides of a processor. Combining GPU processors and memory into a single package maximizes performance-per-watt efficiency. This innovation addresses multiple choke points being experienced today in data processing.
This revolutionary stacking capability not only enables the industry standards for HBM4, which can soon support up to 16 stacked DRAM dies. Hybrid bonding technology is specifically designed to work within a millimeter of GPUs and other AI accelerators. This physical closeness increases data transfer rates exponentially, while dramatically increasing computing power. Implementing this technology isn’t without its challenges. High manufacturing costs and the need for specialized fabrication plants pose challenges to widespread adoption.
“Relief will come from a combination of incremental capacity expansions by existing DRAM leaders, yield improvements in advanced packaging, and a broader diversification of supply chains.” – Shawn DuBravac
Data Center Expansion
There’s almost 2,000 more of these data centers in the planning stages or under construction around the globe. This explosive expansion is increasing the need for memory chips at breakneck speed. Currently, the number of data centers in the world is estimated at some 9,000 such facilities. This large and rapid expansion is being driven by the explosive growth of cloud computing. AI applications are pushing the need for an even greater processing power.
NVIDIA’s data center business revenue underscores this trend, soaring from barely a billion dollars in late 2019 to an astounding $51 billion by the end of October 2025. With the rapid development of large language models and other AI technologies, the need for high-speed data processing will only grow.
As noted by industry insiders, companies might not want to ramp up production capacity in a shaky economy. Storage and memory futurist, Thomas Coughlin, brought up a big one. He noted that businesses often lack the liquidity reserves to invest in new capacities during non-peak periods.
“In general, economists find that prices come down much more slowly and reluctantly than they go up. DRAM today is unlikely to be an exception to this general observation, especially given the insatiable demand for compute.” – Mina Kim
Addressing the supply issues with DRAM involves two primary strategies: innovation and the construction of additional fabrication plants. Hybrid bonding technologies hold the promise of much greater efficiency in memory chips. The industry needs to focus on scaling production and doing so quickly enough to meet accelerating demand.
The company is already ramping up production in a new fab located in Pyeongtaek, South Korea, which began production in 2028. This is an important step, as this company expands its existing capacity. Developments in next generation memory technologies will help alleviate the stress of quickening data center build out. They can help ensure that AI development fosters inclusive economic growth.
While firms continue to push boundaries and find new ways to innovate, they should be focused on the fiscal pressures that come with such aspirational work. Widening production facilities is critically needed to address the persistent chip shortage. Simultaneously, preparing for and promoting the next wave of technological innovation will be key to conquering this challenge.