vLLM Enables Efficient 1M Token Inference with DeepSeek V4 Day-0 Support

vLLMvLLM

· Updated

vLLM has launched immediate support for the DeepSeek V4 Pro and Flash models, which natively handle context windows of up to 1 million tokens. By implementing a new long-context attention mechanism, the update allows developers to serve these massive models with significantly lower memory and compute overhead.

vLLM, an inference engine for serving large language models, added Day-0 support for the DeepSeek-V4 family. This includes the V4 Pro and V4 Flash models, which are Mixture-of-Experts (MoE) architectures (models that activate only a fraction of their parameters per token) supporting 1 million tokens.
Context window
1M tokens
V4 Pro parameters
1.6T total (49B active)
V4 Flash parameters
284B total (13B active)
V4 Pro efficiency
10% KV cache vs V3
V4 Pro compute
27% inference FLOPs vs V3
Availability
Open-weight models on vLLM

This update mirrors the DeepSeek-V4 model family and addresses the hardware barriers of million-token windows. The implementation uses compressed sparse attention to reduce KV cache (memory used to store conversation history) overhead, following a pattern seen in high-performance attention kernels that slash long-context costs for frontier models.

You can now deploy these models on vLLM to process massive datasets without prohibitive compute costs. This is part of a broader shift toward integrating long-context models into agentic workflows. Both are available as open-weight models for immediate self-hosting or integration into production pipelines.

View the full update on vllm.ai
vLLM
vLLM
@vllm_project
X

🎉 Day-0 support for @deepseek_ai V4 Pro and Flash on vLLM — a new generation of DeepSeek model, purpose-built for tasks up to 1M tokens. Alongside the release, we're publishing a first-principles walkthrough of the new long-context attention and how we implemented it in vLLM. https://t.co/ho9pSI4IXs

64retweets448likes
View on X

Still wondering? A few quick answers below.

DeepSeek V4 is a new generation of Mixture-of-Experts language models designed for massive context tasks. It includes two main versions: the flagship V4 Pro and the faster V4 Flash. Both models natively support context windows of up to 1 million tokens, making them suitable for processing large documents or entire codebases in a single interaction.

DeepSeek V4 uses a new attention mechanism that combines Compressed Sparse Attention and Heavily Compressed Attention. This architecture significantly reduces hardware requirements compared to previous models. For a 1 million token context, the V4 Pro model requires only 10 percent of the KV cache and 27 percent of the inference compute power used by the earlier DeepSeek V3.

DeepSeek V4 Pro is the larger flagship model with 1.6 trillion total parameters, activating 49 billion per token. DeepSeek V4 Flash is a smaller, more efficient version with 284 billion total parameters and 13 billion active parameters. While both support 1 million tokens, the Flash version is optimized for lower latency and significantly reduced serving costs.

DeepSeek V4 is released as an open-weight model family, meaning the trained parameters are publicly available for download and self-hosting. Developers can run these models on their own infrastructure using inference engines like vLLM. This allows organizations to maintain data privacy and control while utilizing frontier-level long-context capabilities without relying solely on external API providers.

vLLM provides Day-0 support for DeepSeek V4 by implementing the model's unique long-context attention mechanism directly into its high-throughput inference engine. This integration allows users to serve V4 Pro and Flash models with optimized memory efficiency. Alongside the code release, vLLM published a technical walkthrough explaining the first-principles implementation of this new attention architecture.

Every HeadsUpAI update is written based on its original source and reviewed before it's published. Read our editorial standards →

Share this update