How else can sparse attention be accelerated? LSA makes a single layer of GLM 22.7 times faster under tens of millions of tokens.

Beating AI News Flash: UK-based AI research institute Prefrontal and researchers from ETH Zurich have open-sourced Latent Sparse Attention (LSA), a sparse attention technique for ultra-long contexts.

Models such as DeepSeek, MiniMax, and Zhipu have already adopted sparse attention, processing only a small number of key tokens at a time to reduce computation. However, to identify these tokens, many existing methods still require a lightweight indexer to scan all historical information and score each token individually. In experiments with GLM-5.3-Flash at 10 million tokens, researchers found that 98% of the prefill time for a single layer of sparse attention was spent on filtering information.

LSA first clusters tokens at different positions into groups based on content similarity, then has the indexer score these groups to select relevant tokens. This way, tokens that originally needed to be checked individually can first be filtered by group. The team also attempted to build multi-level indices, allowing the model to progressively narrow the search scope layer by layer without having to scan all groups every time.

The researchers integrated LSA into GLM-5.3-Flash and Qwen-3.8-27B, retaining the original model weights and training only a small number of additional index parameters. On NVIDIA H200, GLM's single-layer prefill is 22.7 times faster than the original sparse attention, and the full model's first-token latency is reduced by approximately 55%. Qwen's single-layer prefill is 72.4 times faster than the original full attention, and the full model's first-token latency is reduced by approximately 89%.

However, the highest speedup results came from the version that still requires scanning all groups. The multi-level index, which can further reduce search volume, showed a significant accuracy drop in GLM's 10-million-token test.

If retrieval can avoid traversing all history while maintaining accuracy, AI will have the opportunity to process tens of millions of tokens of code, literature, and long-term Agent operation logs more efficiently. LSA has currently validated the speedup effect of semantic grouping, but multi-level retrieval still needs further improvement.