DeepSeek and Kimi are both aggressively caching, while NVIDIA has started enabling cross-model cache reuse.

According to Watchful.AI Monitoring, over the past two years, large-model inference optimization has been focusing on KV Cache. It is the intermediate calculation result left after the model reads the context, and the longer the content, the more it consumes GPU memory and bandwidth. Leveraging MLA, DeepSeek-V2 has reduced KV Cache by 93.3% compared to DeepSeek 67B; Kimi Linear has further reduced it by up to 75% compared to full MLA. Everyone is working to make processing long contexts more cost-effective.

But all these solutions face a common challenge: the cache usually can only be reused within the same model. When an agent switches from a small model to a large model, the target model has to recalculate the previous tens of thousands or even hundreds of thousands of tokens. The more frequent the model switching, the harder it is to ignore this redundant computation.

NVIDIA's latest paper aims to break down this barrier. The team discovered a clear linear relationship in the cache between models of the same family with matching KV structures but different sizes. By calibrating text with 500 segments, each consisting of 1024 tokens once, a mapping can be fitted to transfer a precomputed KV Cache from one model to another for continued use.

When transitioning from Qwen3-14B to 32B, recalculating a 32K context takes about 7 seconds, while cache conversion takes only about 0.28 seconds, a speedup of about 25 times. In other words, in the future, even simple tasks can be handled by a small model first, which calculates the long context into a KV Cache; when stronger capabilities are truly needed, the cache can be transferred to a large model, allowing the large model to start processing and generating directly.

This approach will save more computing power in model routing. Currently, using a small model for simple tasks primarily saves on generation costs; however, when switching to a large model, the long context often needs to be recalculated by the large model. If cross-model KV Cache becomes mature, even this prefill part can be handed over to the small model. Agents can maintain the context with an inexpensive model in the long term and only wake up the large model when facing challenges, without having to start from scratch every time.