Eagle has released EAGLE-4, the newest generation of its drafter family for accelerating large language model inference. The core idea is that one drafter adapts its speculative decoding strategy to batch size, hardware and workload while keeping the target model's output distribution.
A small model guesses several tokens ahead and the large model verifies them in one pass. EAGLE-4 uses a sliding-window draft layer, a low-rank candidate head and an append-only draft cache, with both dense and latent-MoE draft structures. The measured pair is Qwen3.8-27B with its EAGLE-4 drafter.
Image: blog.eagle.ai
The repository, eagle-harness, is a pip-installable package that serves drafts on SGLang, vLLM and TokenSpeed through plugins. It sizes the speculative tree from measured draft and verification costs and hooks into engine interfaces without patching the engine at runtime. Patched engines ship as pinned wheels and container images.
In the project's own table, with SGLang on four B300 GPUs at concurrency 8, EAGLE-4 goes from 1,479 to 2,821 tokens per second, 1.91 times the baseline. With vLLM on four B300 at concurrency 1 the gain reaches 2.99 times. On one AMD MI355X, AMD reports about 300 tokens per second for a single BF16 request, 3.5 times plain decoding.
Image: blog.eagle.ai
In the comparison the project shared on X at 512 concurrent requests, EAGLE-4 reaches 28,340 aggregate tokens per second, against 16,641 for DFlash 2, 14,229 for DSpark and 21,203 for the vanilla model. The project reports 1.70 times DFlash 2 and 1.99 times DSpark. These figures are the project's own and have not been independently verified.
The code is on GitHub with the v0.1.0 release and engine wheels, and the Qwen3.8-27B drafter is on Hugging Face.
No comments yet. Be the first!