vLLM v0.28.0 Released with Major Kimi-K3 and DeepSeek V4 Performance Optimizations
Summary
Updated Sep 15 — vLLM's Kimi K3 optimization suite delivers up to 2.8x throughput and 85% TTFT reduction versus v0.27.1 at high concurrency.
- • vLLM v0.28.0 ships 584 commits from 270 contributors, including 76 newcomers to the project
- • Kimi-K3 gets Decode Context Parallel support and fused kernels delivering ~60% better TTFT
- • DeepSeek V4 gains full end-to-end sparse MLA plus ROCm support on gfx11 and gfx950 GPUs
- • Default max batched tokens doubled from 8,192 to 16,384; tiered KV cache disk offloading added
Updates
2.8x throughput at concurrency 16 vs v0.27.1 (8K/1K workload)
Benchmarked with TP8, 8-token DSpark speculation on B300/CUDA 13.3: +180.6% throughput at concurrency 16, +150% at concurrency 4, +120% at concurrency 1.
TTFT reduced 72–85% across concurrency levels vs v0.27.1
TTFT at concurrency 1: -83.4% (2262ms → 376ms); concurrency 4: -72.3%; concurrency 16: -85.3% (7601ms → 1121ms). Average latency down 57–60%.
ReplaySSM reconstructs KDA state instead of storing per-step
Buffers recent SSM inputs and reconstructs accepted state at commit rather than writing T extra states per draft step; rollback moves only a buffer pointer (PR #51855).
Zero-copy mixed KDA batches add 5–8% throughput at scale
Replaced 6 index_select + 2 index_copy_ ops per layer with contiguous zero-copy slices and direct output writes (PR #56159); gains concentrated at concurrency 4 and 16.
Deferred MXFP4 finalization cuts end-to-end latency ~5%
Top-k finalization moved inside latent-tail kernel; removes one kernel launch and avoids writing/rereading one intermediate tensor (PR #53152, init-order fix in #53327).
Details
584 commits from 270 contributors (76 new)
One of the largest contributor counts in vLLM history, reflecting rapid ecosystem growth and broad community adoption of the serving framework.
Kimi-K3: Decode Context Parallel (DCP) support
DCP support combined with fused FlashKDA decode and prefill kernels delivers 1.5–3x kernel-level speedups for Kimi-K3 inference workloads.
~60% better Kimi-K3 DSpark TTFT via adaptive budget
Adaptive speculative token budget significantly reduces time-to-first-token for Kimi-K3, with optional shared-expert sharding saving ~17 GiB of memory per GPU.
DeepSeek V4 sparse MLA end-to-end complete
Sparse MLA now works end-to-end for plain decode, MTP, and DSpark speculative decoding, alongside AMD Quark NVFP4 support and ROCm enablement on gfx11 and gfx950.
Speculative decoding: DFlash2 and confidence-scheduled verification
DFlash2 with local convolution and candidate selector, plus DSpark confidence-scheduled verification and async scheduling auto-enabled for draft models.
Default max batched tokens raised 8,192 → 16,384
Doubles the default batch processing capacity out of the box, directly improving throughput for production serving deployments without manual configuration.
Tiered KV cache: disk offloading now supported
Adds disk-based secondary tier KV cache offloading, out-of-tree secondary tier managers via module_path, CPU canonical layout for parallelism-agnostic offload, and tiering metrics.
Rust frontend: multimodal gRPC, RL lifecycle control
Standalone renderer, multimodal image inference over gRPC, explicit data-parallel rank routing, RL lifecycle control, and protobuf schemas published to Buf.
Model Runner V2: E/P/D disaggregation and weight offloading
Production-matured runner adds encoder/prefill/decode disaggregation, weight offloading, multi-layer MTP KV cache support, encoder CUDA graphs, and thinking token budget support.
Breaking: bitsandbytes moved out-of-tree; Transformers → 5.15.0
bitsandbytes support migrated to an out-of-tree plugin; Transformers bumped to 5.15.0; deprecated calculate_kv_scales removed; override_attention_dtype removed.
Source: vLLM GitHub Release Notes v0.28.0 (via Hacker News, August 29, 2026).
What This Means
vLLM v0.28.0 is a substantial infrastructure release that cements the framework's role as the production backbone for serving today's most capable open-weight models. The targeted optimization packages for Kimi-K3 and DeepSeek V4 — two of the most performance-demanding frontier models — directly reduce inference costs and latency for organizations deploying these models at scale. The doubling of default batch capacity and addition of disk-tier KV cache offloading lower the hardware requirements for high-throughput serving, broadening access. With 270 contributors and 76 newcomers, vLLM's community momentum is accelerating, making it increasingly difficult for proprietary serving solutions to compete on feature velocity in the open-source space.