vLLM v0.28.0 Released: Major Performance Optimizations for Kimi-K3 and DeepSeek V4 Support

Key Takeaways

• Kimi-K3 Performance Breakthrough: Extensive stack-wide optimizations including Decode Context Parallel (DCP) and fused FlashKDA kernels.
• DeepSeek V4 Full Integration: End-to-end support for sparse MLA across plain decode, MTP, and DSpark speculative decoding.
• Substantial Memory Savings: New optional shared-expert sharding reduces memory usage by approximately 17 GiB per GPU.
• Enhanced Hardware Compatibility: Expanded ROCm support for Kimi-K3 and DeepSeek V4, specifically targeting gfx11 and gfx950 architectures.
• Efficiency Gains: Kernel-level speedups of 1.5x to 3x for combined all-gathers and a 60% improvement in DSpark TTFT.

In-Depth Analysis

The Kimi-K3 Performance Push

The release of vLLM v0.28.0 marks a significant milestone for Kimi-K3 users, as the development team has implemented a major optimization effort across the entire software stack. One of the most notable additions is the support for Decode Context Parallel (DCP), which allows for more efficient handling of long-context sequences during the decoding phase. To further enhance throughput, the update introduces fused FlashKDA decode and prefill kernels, which streamline the computational path for attention mechanisms.

Beyond kernel fusion, the update incorporates SiTU activation support for MegaMoE and GEMM-RS for sequence parallelism. These technical improvements are complemented by a 1.5x to 3x kernel-level speedup in combined all-gathers, a critical operation in distributed inference. Perhaps most impactful for user experience is the introduction of an adaptive speculative token budget, which has demonstrated a ~60% improvement in Time to First Token (TTFT) for DSpark. For organizations constrained by hardware, the optional shared-expert sharding is a game-changer, potentially saving 17 GiB of memory per GPU, thereby allowing larger models to run on standard hardware configurations.

DeepSeek V4 and Sparse MLA Support

vLLM v0.28.0 brings robust end-to-end functionality for DeepSeek V4. A core component of this support is the implementation of sparse MLA (Multi-Head Latent Attention), which is now functional for plain decode, Multi-Token Prediction (MTP), and DSpark speculative decoding. This ensures that DeepSeek's latest architectural innovations can be fully leveraged within the vLLM ecosystem.

To support high-performance execution on diverse hardware, the release includes AMD Quark NVFP4 support and specific optimizations for sparse top-k metadata kernels. The development team has also focused on reducing overhead by narrowing eager CUDA graph regions, which helps in stabilizing performance during complex inference tasks. Furthermore, the integration includes reasoning-effort prompts and mappings, allowing for more nuanced control over the model's output generation process.

Hardware Expansion and Speculative Decoding

Hardware flexibility remains a core focus for vLLM. Version 0.28.0 extends ROCm enablement to the gfx11 and gfx950 architectures, ensuring that AMD users can benefit from the latest optimizations. Kimi-K3 is also now compatible with ROCm via the V2 model runner, bridging the gap between NVIDIA and AMD performance capabilities.

Speculative decoding continues to evolve in this release with the introduction of DFlash and other advances. By optimizing the speculative decoding pipeline, vLLM reduces the latency associated with generating high-quality tokens, making real-time interaction with massive models more viable for production environments.

Industry Impact

The release of vLLM v0.28.0 significantly lowers the barrier to entry for deploying state-of-the-art models like Kimi-K3 and DeepSeek V4. By achieving a 17 GiB reduction in memory usage per GPU through shared-expert sharding, vLLM is enabling more efficient utilization of existing data center resources. This optimization is particularly relevant as the industry moves toward Mixture-of-Experts (MoE) architectures, which are traditionally memory-intensive.

Furthermore, the 60% improvement in TTFT and the 1.5-3x speedup in communication kernels signal a shift toward ultra-low-latency inference. As AI applications move closer to real-time human-agent interaction, these stack-level optimizations become the foundation for the next generation of responsive AI services. The expanded support for AMD hardware via ROCm also fosters a more competitive and diverse hardware ecosystem, reducing reliance on a single vendor for high-performance LLM serving.

Frequently Asked Questions

Question: How does vLLM v0.28.0 improve memory efficiency for large models?

Version 0.28.0 introduces optional shared-expert sharding, which can save approximately 17 GiB of memory per GPU. This is particularly beneficial for MoE models like Kimi-K3, allowing them to fit into smaller memory footprints without sacrificing performance.

Question: What specific improvements were made for DeepSeek V4?

vLLM now supports sparse MLA end-to-end for DeepSeek V4. This includes compatibility with plain decode, MTP, and DSpark speculative decoding. Additionally, it adds AMD Quark NVFP4 support and optimizations for sparse top-k metadata kernels.

Question: What are the performance gains for Kimi-K3 in this release?

Kimi-K3 sees a major performance push including a ~60% better TTFT for DSpark due to an adaptive speculative token budget. It also benefits from 1.5x to 3x speedups in kernel-level all-gathers and the introduction of fused FlashKDA kernels.