Starstruk Logo MarkStarstruk

v2.2.20 v0.31.0

high risk

vllm-project/vllmSep 30, 2026 → Oct 5, 20265 days between releases

What's New

Introduces fast restart via `vllm preload` CLI and performance optimizations for DeepSeek-V4.1-Flash.

Breaking Changes

  • Per-request `mm_processor_kwargs` and `media_io_kwargs` now require `--trust-request-mm-kwargs`.
  • The `tokenizer_mode='slow'` option has been removed.
  • `--enable-mamba-fine-grained-prefix-cache` was renamed to `--enable-mamba-shared-prefix-checkpoint`.
  • Online quantization via `quantization='fp8'` now redirects to `fp8_per_tensor`.
  • The AllSpark INT8 W8A16 backend was removed.

New Features

  • `vllm preload` CLI for fast engine restarts with weight caching and readiness endpoints.
  • Model Runner V2 supporting draft-model speculative decoding (e.g., LiLiCorr drafter) and async scheduling.
  • New MoE backends: MoonEP and DeepEPv2 with sequence parallelism and EPLB improvements.
  • New model capabilities: DiffusionGemma structured generation, MiMo V2 MXFP4 MoE, and Granite 4.2 parser.
  • New scheduling controls: `--max-num-active-seqs` and adaptive `--long-prefill-token-threshold`.

Improvements

  • DeepSeek-V4.1-Flash performance improvements including FlashMLA mega attention and NVFP4 compressed KV cache.
  • Fast restart mechanism with weight-cache daemon and CRIU-based initialized-engine snapshots.
  • Model Runner V2 enhancements featuring custom logits processors.
  • Large-scale serving improvements via MoonEP and DeepEPv2 backends.
  • Enhanced security with per-request multimodal kwargs gating and improved prefix cache tagging.

Upgrade Notes

Update configuration to remove `tokenizer_mode='slow'` and update mamba prefix cache flags. Ensure `mm_processor_kwargs` usage includes the `--trust-request-mm-kwargs` flag.

The Two Releases

v2.2.20

v2.2.20

Sep 30, 2026· 22 days ago

This release introduces new AI employee features that expose referenced knowledge base documents and fixes several bugs related to data sources, UI interactions, and workflow approvals.

Full release notes
v0.31.0

v0.31.0

Oct 5, 2026· 17 days ago

This release introduces significant performance optimizations for DeepSeek-V4.1-Flash and a new fast restart mechanism via the `vllm preload` CLI. It also enhances large-scale serving capabilities with new MoE communication backends and advanced features in Model Runner V2.

Full release notes