v1.23.0 v0.30.0
Osmantic/ODSSep 22, 2026 → Sep 22, 20260 days between releases
What's New
Major version jump introduces LLM inference engine replacing PDF text processing capabilities
Breaking Changes
- Scale-out endpoints are opt-in via `--enable-scale-out`
- `g_idx` parameter removed from GPTQ activation ordering
- `all` Mamba cache mode deprecated
- `python -m vllm.entrypoints.grpc_server` deprecated
- YaRN aliases removed
New Features
- New models: DeepSeek-V4.1-Flash, GLM-5.3-Flash, K2-Horizon, Cohere Compass
- Fast Start IPC caching with persistent GPU weight-cache daemon
- Model Runner V2 with FULL CUDA graphs and MTP speculative decoding
- Watermarking with Gumbel-max generation and keyed PRF
- HiSparse host-resident tier for sparse-MLA decode
Improvements
- CUDA graph optimizations for performance
- Dual-batch overlap in Model Runner V2
- Persistent GPU weight-cache daemon for IPC loading
- Async Engram prefetch with Engram DP sharding
- MXFP8 support through FlashMLA V4.1
Upgrade Notes
Enable scale-out endpoints via the `--enable-scale-out` flag and remove references to deprecated `g_idx`, `all` Mamba cache mode, gRPC server, and YaRN aliases. Update weight loading scripts to use `--load-format ipc_cache` to leverage Fast Start caching.
The Two Releases
1.23.0 — Text-layer repairs
Sep 22, 2026· 1 month ago
Enhances text-layer repairs to handle complex layouts, invisible text layers, and right-to-left text correctly.
Full release notesv0.30.0
Sep 22, 2026· 1 month ago
A major performance and feature release introducing new models, Fast Start caching, HiSparse support, and Model Runner V2 with advanced CUDA graph optimizations.
Full release notes