EAGLE-4 boosts Qwen's generation speed by 3.6 times, and even mobile CPUs can achieve a 3x speedup.

Beating AI News Flash: Large model inference acceleration company Eagle has released EAGLE-4, an upgrade to its speculative decoding solution. The new version focuses on reducing the computational overhead of the draft model and can adjust the prediction scale based on hardware performance and concurrency levels, avoiding additional prediction computation that could actually slow things down.

On 4 NVIDIA B300 GPUs, Qwen3.8-27B's single-request generation speed increased from 240.5 tokens per second to 863.8 tokens per second, reaching 3.6x that of standard inference. With 512 concurrent requests, total throughput increased from 21,200 tokens per second to 28,300, a 34% improvement.

Mobile devices also saw significant gains. On the Motorola Razr 70 Ultra, Gemma-4-E2B-it achieved 3.2x faster long-text generation using only the CPU, reducing the time for the same task from 43 seconds to 14 seconds, with energy consumption per token reduced by 53%.

EAGLE-4 has been open-sourced and supports integration with SGLang, vLLM, and experimental TokenSpeed. However, unlike the previous three generations released under the Apache 2.0 open-source license, the new version adopts a restrictive commercial license. Enterprises with combined revenue exceeding $20 million over 12 consecutive months, as well as companies offering competing inference services, require separate authorization for commercial use.