AI Semiconductor Endgame: Will the Shortage Last at Least Another Five Years?

Original Title: "AI Semiconductor Endgame 2026(II)"
Original Author: fin, AI Analyst

As the semiconductor structure evolves into the AI inference mainstream, memory and storage have become the biggest bottlenecks. The biggest doubt in the market about memory and storage is:

Will HBM/DRAM/SSD break free from traditional cyclicality?

Will the GPU architecture roadmap dependent on HBM's exponential growth stop evolving? When will it stop?

How significant is ChangXin's capacity expansion? Will it drag this market back into a cyclical quagmire?

This article attempts to establish a framework to address these questions

Everything is cyclical, and memory's cyclicality is particularly strong, mainly due to the excessively long production cycle, leading to mismatches between rapid production expansion and periods of demand scarcity.

Possible ways to break free from traditional cyclicality

1. Customization: Products are not interchangeable, capacity cannot be easily reallocated, requiring long-term contracts.

2. Structural exponential demand growth: The demand curve itself is very steep, and the supply has always lagged behind.

3. Rapid technological iteration: Each generation of products rapidly outdates the previous one.

Fulfilling any one of these can partially break free from traditional cyclicality; fulfilling two to three can break free from most traditional cycles.

According to this framework, HBM satisfies about two and a half of these criteria

1. Customization Requires Long-Term Contracts (Relatively Weak, Counts as Half)

Indeed, HBM does have elements of customization and Nvidia codesign, but it is not very strong. The truly customized parts are only in the packaging and base dies, while the dozen or so layers of DRAM dies on top are still fully JEDEC standardized.

For example, when Samsung's HBM3E did not pass NVIDIA's qualification and its market share plummeted from about 60% to 20%, it did not hold onto that batch of production capacity as scrap but instead quickly supplied it to Google's TPU and AMD. Physically, the HBM3E provided to NVIDIA and AMD is the same thing. Therefore, production capacity can still be partially reallocated freely.

Customization has become more prevalent post-HBM4, including integrating custom logic and/or cache on the base die. A more sophisticated approach is to directly place an HBM4E memory controller and custom die-to-die interface into the logic base die.

SemiAnalysis mentioned that OpenAI, NVIDIA, and AMD are each working on custom HBM, but this refers to customization of the base die, with the DRAM layer above remaining standard.

With partially customized features, HBM mainly requires collaboration in packaging, leading customers to sign long-term contracts. However, capacity can indeed be transferred, making HBM somewhat feasible.

2. Structural Exponential Demand Growth (Fulfilled)

The most straightforward reason is the hardware upgrade demand for Nvidia's token factory token throughput, which has rapidly driven the upgrade and generational shift of HBM bandwidth, as well as the exponential growth in HBM size demand.

This point is actually the conclusion of the previous article "AI Semiconductor Endgame 2026(I):"

token throughput = HBM size × HBM bandwidth, doubling each generation.

HBM size per GPU has been growing by over 40% annually.

The steepness of this demand curve, with DRAM supply-side wafer growth of 14% multiplied by a 9% density improvement, is difficult to catch up with.

In the hardware field, due to the high-bandwidth and large memory size requirements of the attn stage KV cache, HBM has also achieved a unique status. Even if HBM prices increase three to five times, the marginal token throughput increase from investing in HBM is still much more cost-effective than spending elsewhere.

Several other memory paths, including SRAM, HBF, CXL, and PIM, are currently unable to compete head-on in the main HBM track of KV cache/attention. For at least the next five years or even longer, it is unlikely that alternative paths will be found.

3. Rapid Technology Iteration Upgrade (Fulfilled)

The DDR3 era lasted for 15 years and we are still only in the DDR5 era. In contrast, the upgrade and generational shift speed of HBM is basically one generation every two years, much faster than traditional DDR, and there is even an accelerating trend recently. HBM size x HBM BW doubles each generation, which is currently completely following this pattern.

With an HBM upgrade every two years, the NV GPU speed has essentially increased exponentially: 2TB/s -> 3.5TB/s -> 4.8TB/s -> 8TB/s -> 22TB/s. The speed of HBM is directly proportional to the inference token throughput, making it economically impractical to use the previous generation of HBM due to diminishing returns. There is a strong incentive for everyone to use the latest products as much as possible, even though they are more expensive, because they provide greater benefits (token throughput).

The logic of the Token Factory era is that the more technological upgrades (HBM bandwidth) there are, the more profit can be generated.

This speed difference has led to a situation similar to CPUs: old products rapidly depreciate in value, causing the value of hoarded goods to decrease. For example, the value of HBM3 has depreciated very quickly, and mainstream products today basically do not use it anymore.

The rational choice for HBM manufacturers has shifted from competing in quantity by maximizing current production capacity to instead competing in stability and HBM speed by focusing on technology. They are now competing for the qualification share of the next generation on the NVIDIA platform (quality competition), thus avoiding the prisoner's dilemma of everyone being unwilling to reduce production and lose market share during a traditional downturn.

When comparing HBM to traditional DRAM, if two and a half out of three conditions are met, can HBM break free from the traditional cyclical trend?

The source of memory cyclicity is commonly explained by the Commodity nature of DRAM (lack of differentiation -> price wars -> stockpiling). However, the Commodity nature itself does not create cycles; it merely amplifies amplitudes.

In the DRAM sector, a prisoner's dilemma has occurred before. During a downward cycle, Samsung once expanded production to gain market share, leading to a situation where whoever reduces production first suffers losses. This results in no one daring to reduce production easily, and ultimately, everyone suffers heavy losses.

In reality, the main structural source of cyclical trends is the excessively long supply cycle, which easily becomes misaligned with the demand cycle. Building a fab takes 3 years and requires investments of billions of dollars, and once the decision is made, it is irreversible. Demand growth can be unstable, with each new paradigm of growth, such as cloud services, mobile internet phones, and online demand during a pandemic, experiencing explosive growth. However, growth slows down after two years, supply exceeds demand, prices drop sharply, and a loss-making cycle ensues.

Everything is cyclical, and HBM is no exception. As long as the token demand continues to grow exponentially, structural exponential growth will dampen the cyclicality, as the demand becomes more predictable. Additionally, when there is a price drop, customers have a greater need to increase their HBM size (thereby increasing token throughput). Furthermore, the somewhat customized nature of HBM leads to longer lead times, transitioning from cyclical to growth cyclicality, making this cycle particularly long.

· Cyclical: Earn more in upswings, lose more in downswings

· Growth Cyclicality: Earn more in upswings, earn less in downswings

Moreover, in addition to these three conditions that break free from the traditional cycle, HBM/DRAM has one key advantage:

4. Because DRAM density scaling is slowing down, and HBM upgrades lead to an increase in the stacking factor of DRAM, the difficulty of supply-side expansion continues to rise

Around the year 2000, the DRAM bit density per wafer was growing by approximately 45% each year, meaning that even if the wafer count did not increase, the annual supply of DRAM bits could still grow by 45%.

A decade ago, the annual growth rate of DRAM bit density dropped to 20%, and now the growth rate has dropped to 9%. Previously, DRAM expansion hardly required new fabs to achieve a 20-30% annual increase in bit volume; now, expanding DRAM production relies more on increasing wafer counts, which involves constructing new fabs and clean rooms.

Another challenge in rapidly expanding HBM production is that HBM3e requires roughly three times the number of DRAM wafer wafers, while HBM4, due to increased stacking density, requires roughly four times the number of DRAM wafer wafers. This means that manufacturing HBM bits has become increasingly difficult relative to DRAM bits; the number of HBM bits produced per DRAM wafer is decreasing, resembling a deflationary trend.

Will HBM eventually revert from growth cyclicality to traditional cyclicality? The most critical factor is structural exponential growth, so

In the era of AI inference, will the GPU architecture roadmap that relies on HBM's exponential growth stop evolving? When will it stop?

token throughput = HBM size × HBM bandwidth. The primary reason for the exponential growth of HBM size in the first principles of this HBM Index Growth is actually the growth of the KV cache. The characteristics of the KVCache and Attention are very much in line with HBM. This even puts HBM ahead of other technological paths, maximizing the utilization of KVCache and the Attention phase.

In other words, if the KV cache does not exist from an architectural perspective, then the logic of HBM size exponential growth will also be challenged.

So the essence of this issue is actually whether this round of attention mechanisms represented by Transformer, and the KV cache mechanism derived from it, will disappear. Will they be replaced after the ebb?

Historically, in each AI model architecture revolution, what has been truly retained are those primitive operations that have some universality in mathematics.

For example, FFN (Feedforward Neural Network, which is abundant in MLP layers in the model) was a product of the deep learning era in 2012, but it has survived all the way to today's large language models, still occupying a significant portion of the model's parameters. Why can it survive? Because this is also a kind of universal approximation theorem: any wide enough MLP can approximate any continuous function.

Attention is most likely also such a primitive that will be retained. Because it addresses an equally fundamental issue: dynamic routing between any two positions in a sequence, allowing any two positions in a sequence to establish connections as needed. Once this ability is proven effective, it is hard to discard.

Therefore, even if future architectures evolve from pure Transformers to hybrid architectures or world models, the attention layer will still exist, the KV cache (or its equivalent after latent compression) will still be needed, HBM will still be one of the inference cores, and the GPU KV cache architecture route relying on HBM Index Growth will not stop evolving.

What about DRAM? Will it be able to break away from traditional periodicity in the future?

HBM has achieved a certain consensus in the market in terms of breaking away from cyclicality, but DRAM has not, and there is currently little consensus in the market.

Returning to the previous framework, among the three conditions for breaking away from the traditional cycle, DRAM is not customized, so we can only look at the pace of technological iteration. The most crucial factor is whether there is structural exponential growth, and the answer is yes.

In the concept of the AI token factory, the primary driver of structural exponential growth is indeed HBM. However, things started to change after the end of 2025: as agentic CPUs began to unleash their potential, the part of DRAM demand associated with the CPU is becoming a new source of structural exponential growth for DRAM.

This growth logic is twofold: the first layer is the rapid expansion of the CPU server TAM, and the second layer is the rapid growth of DRAM usage per CPU core due to agentic flow.

The rapid growth of the server CPU TAM is detailed in the CPU special issue in April, with four key logics, in brief:

1. In AI accelerator clusters, the CPU-to-GPU ratio has shifted from the traditional 1:4 to 1:2 and may even move towards 1:1.

2. In the agentic flow, the proportion of delays in CPU processing is very high, with 50-90% becoming a significant bottleneck, requiring synchronous capacity expansion.

3. AI coding has greatly improved the efficiency of software development engineers (SDEs), with a magnitude increase in code volume. Software API call volume has increased exponentially, directly translating into an exponential increase in CPU hours.

4. To ensure data security and isolation, sandboxes such as the Analytical Agent need to replicate large amounts of databases and user contexts for each task, leading to significant waste of memory (DRAM) and CPU cores. This waste issue is expected to persist for five years or even longer. Additionally, it is technically challenging to reduce CPU hours through optimization.

That is why two quarters ago, AMD's financial report stated that the CPU TAM would reach 60B by 2030. Two months ago, AMD/ARM doubled their 2030 forecast for CPU TAM to 120B. One month ago, Nvidia once again doubled its 2030 forecast for CPU TAM to 200B.

And last week, Bernstein once again raised the 2030 CPU TAM guidance to 223B. In my view, it is almost certain that the 2031 CPU TAM will be revised upwards to 400B. The only suspense is when will the major players announce the revision of this guidance.

Layer 2, why is the amount of DRAM per server CPU core rapidly increasing in the agentic era?

1. An Agent is a stateful long-running process, not a stateless request-response

Traditional web/SaaS is stateless: requests come in, memory is allocated, processed, and then immediately reclaimed. On the other hand, an Agent task can run from one minute to one hour, during which time its message history, system prompt, working memory, long-term memory, and tool results buffer all reside in DRAM.

Similar to CPU hours, the memory footprint of each task is challenging to compress due to the requirements of being stateful and the sandbox isolation (each task replicates databases and contexts).

2. The context window is exponentially lengthening, and the working set of each session is expanding, multiplying concurrency by the single-session memory footprint, resulting in a magnification effect

The context window is expanding from 32K → 256K → 1M, the sequence length for reasoning/test-time compute is exploding, and it will continue to grow in the future. The messages resident in each active session increase linearly with the context length.

Now, multiply these two layers together.

Layer one, the CPU server TAM, is expected to increase by about 5–7 times in the 2030–2031 timeframe (60B → 120B → 200B → 223B, and I believe it will reach 400B).

Layer two, the DRAM-to-CPU ratio per core, is roughly increasing by 3–4 times (4~8GB → 16~32 GB/core), but most of this growth may be a one-time windfall.

Multiplying two independent variables, the server-side DRAM demand undergoes an exponential growth

By 2030, even with a conservative $300 billion CPU Total Addressable Market (TAM), at a price of $50 per CPU core, considering the most conservative 16GB/core in the agent era, the calculated minimum incremental amount is 96EB. This is far more than the total DRAM production capacity of 47EB this year, which will barely reach 60EB next year. This represents a significant and astonishing increase.

Although the agentic CPU's DRAM exponential growth is to a large extent a one-time dividend at the second layer, its duration will be long-lasting because the shortage gap is just too vast.

Returning to the framework mentioned at the beginning of the article, among the three conditions for breaking away from the traditional cycle, the first condition of customizing DRAM can be essentially ignored.

As for the second condition: a structural exponential demand source that is challenging to reverse is established. Commodity DRAM now also partially qualifies to break away from the traditional cyclical pattern. While not as thorough as HBM (two and a half), it has already undergone substantive changes.

The third condition, the pace of technological iteration, the rhythm of DRAM is also different from before.

While the pace of previous DRAM technological iterations was heavily dependent on consumer electronics, the progression of DDR was not very useful for performance. However, in the foreseeable future, the traditional DDR consumed by carbon-based consumers will be far less than the consumption of silicon-based consumers (CPU servers) in DRAM.

The marginal utility brought about by the speed upgrade of previous DRAM was very low. But now, due to the increased demand for memory by CPU servers and the increased speed requirements of DDR by edge-side AI, such as Apple's need for faster LPDDR to run local large models, the marginal utility of speed upgrades has increased significantly. Therefore, the speed iteration requirements for DDR6 and LPDDR6 have increased much more than before, as shown in the graph, with the iteration time for LPDDR6/DDR6 shortened and the speed slope starting to rise again.

Previously, when a new generation of DDR/LPDDR technology was launched, everyone's reaction was quite lukewarm, waiting to use it only after the price dropped.

Now, with the introduction of LPDDR6, every company is eager to adopt it as early as possible because the performance gains brought about by the speed increase are within reach.

Furthermore, DDR's supply is additionally taxed by HBM. The rapid expansion of HBM's production capacity each year results in a batch of wafers that could have been used for commodity DDR being diverted to produce HBM. The conversion ratio for HBM is very low. It takes about 3 DDR wafers to produce the same amount of bits as HBM3E, and 4 wafers for HBM4. Therefore, about 3% to 5% of DDR bit growth each year is directly consumed by this HBM bit tax.

So although the DRAM bit volume is expected to grow by around 24% per year in the future (14% from wafer growth and 9% from DRAM node density growth per wafer), when factoring in the HBM bit tax, the annual bit growth of traditional non-HBM commodity DDR is only about 20% (approximately 10% wafer growth × about 9% node density increase)

How significant is Yangtze Memory's expansion? If they massively expand without restraint, could they drag this market back into a cyclical downturn?

Yangtze Memory's expansion in recent years has been quite rapid. By 2025, they will be producing 200,000 wafers per month, and by 2026, the contribution from the Beijing fab and additional production lines will increase this to 320,000 to 350,000 wafers per month.

The Phase 1 and Phase 2 of the Shanghai plant that is currently under construction is expected to add 100,000 wafer capacity per month by 2027 for Phase 1 and another 100,000 wafer capacity per month by 2028 for Phase 2. In other words, there will be 420,000 wafers per month in 2027 and 500,000 wafers per month in 2028.

However, it is important to note that Yangtze Memory's DRAM bit density is only about half of the top three manufacturers. Therefore, with 500,000 wafers per month, Yangtze Memory can only produce half of the dram bit volume output compared to other manufacturers. When calculating the wafers per month, it is assumed to be half for equivalency.

After applying this discount, Yangtze Memory's impact on the entire DRAM industry is significantly reduced. From the end of 2025 to the end of 2028, Yangtze Memory's impact on DRAM bit capacity CAGR is only about 1.5%, and the overall DRAM capacity CAGR of the industry is estimated to increase from 12.7% to 14.2%.

· DRAM Monthly Capacity (kwspm) 2025E → 2028E CAGR

· Samsung 685K → 920K 10.3%

· SK Hynix 519K → 725K 11.8%

· Micron 340K → 560K 18.1%

· Rest of World Excluding China 150K → 218K 13.3%

· China (Density Halved) 117K → 274K 32.8%

· Total Including China 1811K → 2697K 14.2%

· Excluding China Total 1694K → 2423K 12.7%

Even if ChangXin can maintain its production growth rate in the future, the impact on the industry's annual DRAM bit volume production CAGR by 2030 is unlikely to exceed 3%. It would shift from a 20% CAGR to a 23% CAGR at most

Additionally, ChangXin is limited by lithography, while DDR6 requires higher speeds (starting at 14400 MT/s) and higher density. The top three manufacturers of DDR6 will most likely use a 1x node or more advanced node (~12nm and below) that fully utilizes EUV technology. ChangXin may be constrained in speed and achieve only half the density in DDR6.

Even in a growth cycle, why will this super cycle of DRAM last a long time, at least five years?

The first reason is the structural DRAM demand growth driven by the massive increase in CPU server demand mentioned earlier. When combined with the industry's bit volume CAGR of around 20% on the DRAM supply side, it is clear why the DRAM supply-demand gap will continue to widen over the next few years:

The traditional DRAM supply side, excluding HBM, is expected to grow by about 20% annually. On the demand side, based on a 60B CPU TAM in 2026, with each CPU consuming an average of 8GB/core, priced at $30~35 per core, the demand is 16EB

By 2030, based on a 400B CPU TAM, with each CPU consuming an average of 16GB/core, priced at $80 per core (CPU prices doubling), the demand will be 80EB. The CAGR of this portion of DRAM growth is approximately 50%, far exceeding current estimates

Unlike HBM, which is directly linked to token throughput and thus directly linked to GPU revenue efficiency, the impact of insufficient DRAM on the agent flow is mainly speed-related. For example, comparing 8GB/core to 16GB/core, some workloads may experience a 30% speed reduction, while some low-value tasks can tolerate delays. The motivation for structural growth is strong, but the demand is not as rigid as for GPUs.

Semianalysis stated that this year's DRAM shortfall is in the single digits percentage-wise, while next year it will exceed 10%. Considering the structural nature of the DRAM shortage caused by the surge in agent CPU numbers, this gap will continue to widen each year, with no possibility of reduction before 2030

Another key reason why DRAM can sustain its strength for a long time is that after the price increase, the demand that was eliminated by the price hike did not truly disappear; it was simply delayed, as there is a large reservoir of demand.

The so-called reservoir refers to the "potential demand that will be immediately released once memory prices drop." Their existence means that even if supply catches up periodically, prices are unlikely to collapse because new demand will always emerge from the reservoir to absorb the excess supply:

Memory for Computing Power/Speed is one such reservoir:

There is a significant amount of demand that originally required additional memory to optimize speed and computing power but was suppressed when memory prices were high. This demand will be unleashed once memory prices fall.

For example, Nvidia's CPX prefill acceleration was originally designed to use additional low-cost GDDR7 to create a dedicated prefill accelerator. However, LPDDR/GDDR pricing was too high, even more expensive than HBM before the price hike, making the ROI of this solution unfeasible. However, when mainstream memory prices drop, similar optimization solutions like CPX will make a comeback.

Low-value tasks represent another reservoir: when memory prices rise, causing token prices to remain high, high-value tasks are prioritized while low-value tasks are postponed. Once memory prices decrease, these delayed demands resurface.

Edge AI serves as a reservoir: the memory configuration of AI PCs may have increased from 24GB to 128GB. Apple has explicitly stated that the latest full-version edge AI requires an upgrade from 8GB to 12GB of memory.

Mainstream consumer electronics, agent PCs, and budget smartphones, whose demand decreased due to memory price hikes, all belong to this reservoir.

With all these reservoirs combined, they form a substantial demand cushion. This is why DDR's structural growth in this cycle will have stronger momentum than the market imagines.

Another reason why DRAM prices are unlikely to experience a significant drop is that HBM and DRAM production capacities can be converted interchangeably, leading to a simultaneous re-rating of the entire DRAM complex.

During the upcycle, the profit margin of DRAM significantly exceeded that of HBM, to the extent that the increase in HBM prices was even driven by DRAM. The price of the newly contracted HBM4 this year is the current DRAM price multiplied by 4, reflecting the standard stacking multiplier corresponding to the price of HBM4.

Once DRAM prices drop and gross margin decreases, HBM's long-term transparency ensures profitability. HBM indirectly takes away more DRAM production capacity, and the price reduction of HBM will also motivate GPU manufacturers to upgrade the HBM size as much as possible, indirectly safeguarding the price floor of DRAM.

There is a structural demand growth in DRAM, but with density scaling slowing down, the difficulty of capacity expansion is increasing. Manufacturers' expansion plans are very cautious, and the impact brought by ChangXin in recent years is also limited. In addition, the reservoir of demand is very large. These four reasons have led to the foreseeable future, at least five years or even longer, where DRAM is unlikely to enter a cyclical low.

Can NAND SSDs Break Free from Traditional Cyclicality?

The structural growth momentum of NAND is not as strong as DDR. The main reason for this year's shortage is that several major players have maintained good production discipline and have not significantly expanded production. Annual capacity increases mainly come from technological improvements: an increase in the number of NAND stack layers.

The first structural growth comes from AI, mainly from KV cache offloading, unloading warm/cold KV cache overflow from HBM to NAND SSDs.

But the amazing thing is that this KV cache offloading growth has not even occurred on a large scale, and SSDs are already more severely short in supply than DRAM, with prices rising even more than DRAM. When Rubin CMX ramps up next year, coupled with large-scale application of KV cache offloading, the shortage of SSDs will also grow due to this structural growth.

The second one, another anticipated structural increment from AI video mentioned in the year-end summary last year, has already shown a trend this year.

Seedance's volume is growing at a rate of ten to forty times a year. Currently, it is stuck in the stage where there is insufficient card shortage, and the demand is not fully released due to the lack of computing power. However, once the card shortage stage is over, the structural demand growth for NAND storage from AI videos will continue for quite a long time.

The third structural growth also comes from the exponential growth in Sandbox usage brought about by agent flow. To ensure data security and isolation, the Analytical Agent needs to replicate a large amount of databases and user context for each task, leading to severe waste of memory (DRAM) and CPU cores, which will also bring a significant waste (demand) of SSDs.

The fourth structurally transformative growth, which may take effect after 2030, comes from the HBF route that requires the use of an SSD. It has been highly anticipated in many investment bank analyses. However, this technological route is still distant. Its main role can only serve as a storage for large model weights, where the weights are written once and then read-only. It must be packaged together with a GPU/HBM (48TBps/96TBps); otherwise, it cannot be used effectively due to the slow speed of PCIE7/8. It can only be said that the future holds promise. A more detailed analysis will be provided in the next article on the ultimate evolution of AI semiconductors in 2026 (III).

In summary, the structurally transformative growth of NAND SSDs is not as strong as that of HBM, but it is advantageous in terms of cost-effectiveness. The price is expected to be only $0.8/GB by 2027, which is one fortieth of the price of DRAM at the same time. Therefore, it can be considered a versatile component in the multi-level cache, given its wide range of sources for structural growth.

In other words, there is no scenario where DRAM/HBM prices rise while SSD prices remain stable. If such a situation were to occur, people would find ways to use SSDs to fulfill some functions of DRAM/HBM at a lower cost to achieve similar effects. HBM, DRAM, and NAND are not three separate stories but rather the structural growth of the same AI memory hierarchy at different temperature levels.

With the demand for structurally exponential growth, has NAND SSD finally broken free from its cyclic nature? This will depend on the production discipline of NAND SSD manufacturers. The only possible entity that might not adhere to production discipline would be the long-term storage. After all, this is a prisoner's dilemma situation. Once a player resorts to unethical practices and aggressively expands production, the entire NAND industry will face a much simpler production expansion than DRAM.

At the very least, this round of NAND growth is also a supercycle. The demand brought about by a few exponential structural indices delays the downturn period until 2030, which is not a significant issue.

Original Article Link