v0.12.0

# vLLM v0.12.0 Release Notes Highlights

## Highlights

This release features 474 commits from 213 contributors (57 new)！

**Breaking Changes**: This release includes PyTorch 2.9.0 upgrade (CUDA 12.9), V0 deprecations including `xformers` backend, and scheduled removals - please review the changelog carefully.

**Major Features**:
* **EAGLE Speculative Decoding Improvements**: Multi-step CUDA graph support (#29559), DP>1 support (#26086), and multimodal support with Qwen3VL (#29594).
* **Significant Performance Optimizations**: 18.1% throughput improvement from batch invariant BMM (#29345), 2.2% throughput improvement from shared experts overlap (#28879).
* **AMD ROCm Expansion**: DeepSeek v3.2 + SparseMLA support (#26670), FP8 MLA decode (#28032), AITER attention backend (#28701).

### Model Support

* **New model families**: PLaMo-3 (#28834), OpenCUA-7B (#29068), HunyuanOCR (#29327), Mistral Large 3 and Ministral 3 (#29757).
* **Format support**: Gemma3 GGUF multimodal support (#27772).
* **Multimodal enhancements**: Qwen3 Omni audio-in-video support (#27721), Eagle3 multimodal support for Qwen3VL (#29594).
* **Performance**: QwenVL cos/sin cache optimization (#28798).

### Engine Core

* **GPU Model Runner V2 (Experimental)** (#25266): Complete refactoring of model execution pipeline:
  - No "reordering" or complex bookkeeping with persistent batch removal
  - GPU-persistent block tables for better scalability with `max_model_len` and `num_kv_groups`
  - Triton-native sampler: no -1 temperature hack, efficient per-request seeds, memory-efficient prompt logprobs
  - Simplified DP and CUDA graph implementations
  - Efficient structured outputs support

* **Prefill Context Parallel (PCP) (Preparatory)** (#28718): Partitions the sequence dimension during prefill for improved long-sequence inference. Complements existing Decode Context Parallel (DCP). See RFC #25749 for details.

* **RLHF Support**: Pause and Resume Generation for Asynchronous RL Training (#28037).

* **KV Cache Enhancements**: Cross-layer KV blocks support (#27743), KV cache residency metrics (#27793).

* **Audio support**: Audio embeddings support in chat completions (#29059).

* **Speculative Decoding**:
  - Multi-step Eagle with CUDA graph (#29559)
  - EAGLE DP>1 support (#26086)
  - EAGLE3 heads without `use_aux_hidden_states` (#27688)
  - Eagle multimodal CUDA graphs with MRoPE (#28896)
  - Logprobs support with spec decode + async scheduling (#29223)

* **Configuration**: Flexible `inputs_embeds_size` separate from `hidden_size` (#29741), `--fully-sharded-loras` for fused_moe (#28761).

### Hardware & Performance

* **NVIDIA Performance**:
  - **Batch invariant BMM optimization**: 18.1% throughput improvement, 10.7% TTFT improvement on DeepSeek-V3.1 (#29345)
  - **Shared Experts Overlap with FlashInfer DeepGEMM**: 2.2% throughput improvement, 3.6% TTFT improvement at batch size 32 (#28879)
  - DeepGEMM N dim restriction reduced from 128 to 64 multiplier (#28687)
  - DeepEP low-latency with round-robin expert placement (#28449)
  - NVFP4 MoE CUTLASS support for SM120 (#29242)
  - H200 Fused MoE Config improvements (#28992)

* **AMD ROCm**:
  - DeepSeek v3.2 and SparseMLA support (#26670)
  - FP8 MLA decode support (#28032)
  - AITER sampling ops integration (#26084)
  - AITER triton attention backend (#28701)
  - Bitsandbytes quantization on AMD GPUs with warp size 32 (#27307)
  - Fastsafetensors support (#28225)
  - Sliding window support for AiterFlashAttentionBackend (#29234)
  - Whisper v1 with Aiter Unified/Flash Attention (#28376)

* **CPU**:
  - Paged attention GEMM acceleration on ARM CPUs with NEON (#29193)
  - Parallelize over tokens in int4 MoE (#29600)
  - CPU all reduce optimization for async_scheduling + DP>1 (#29311)

* **Attention**: FlashAttention ViT support, now default backend (#28763).

* **Long Context**: Optimized `gather_and_maybe_dequant_cache` kernel for extremely long sequences (#28029).

* **Multi-NUMA**: Enhanced NUMA functionality for systems with multiple NUMA nodes per socket (#25559).

* **Docker**: Image size reduced by ~200MB (#29060).

### Quantization

* **W4A8**: Marlin kernel support (#24722).
* **NVFP4**: 
  - MoE CUTLASS support for SM120 (#29242)
  - TRTLLM MoE NVFP4 kernel (#28892)
  - CuteDSL MoE with NVFP4 DeepEP dispatch (#27141)
  - Non-gated activations support in modelopt path (#29004)
* **AWQ**: Compressed-tensors AWQ support for Turing GPUs (#29732).
* **LoRA**: FusedMoE LoRA Triton kernel for MXFP4 (#29708).
* **Online quantization**: Moved to `model.load_weights` (#26327).

### API & Frontend

* **Responses API**: 
  - Multi-turn support for non-harmony requests (#29175)
  - Reasoning item input parsing (#28248)
* **Tool Calling**:
  - Parsed tool arguments support (#28820)
  - `parallel_tool_calls` param compliance (#26233)
  - Tool filtering support in ToolServer (#29224)
* **Whisper**: `verbose_json` and `timestamp` features for transcription/translation (#24209).
* **Sampling**: Flat logprob control moved from env var to `SamplingParams` (#28914).
* **GGUF**: Improved HuggingFace loading UX with `repo_id:quant_type` syntax (#29137).
* **Profiling**: Iteration-level profiling for Torch and CUDA profiler (#28987).
* **Logs**: Colorized log output (#29017).
* **Optimization Levels**: `-O0`, `-O1`, `-O2`, `-O3` allow trading startup time for performance, more compilation flags will be added in future releases (#26847)

### Dependencies

* **PyTorch 2.9.0** with CUDA 12.9 (#24994) - **Breaking change** requiring environment updates.
* **xgrammar**: Updated to 0.1.27 (#28221).
* **Transformers**: Updated to 4.57.3 (#29418), preparation for v5 with `rope_parameters` (#28542).
* **XPU**: torch & IPEX 2.9 upgrade (#29307).

### V0 Deprecation & Breaking Changes

**Removed Parameters**:
* `num_lookahead_slots` (#29000)
* `best_of` (#29090)
* LoRA extra vocab (#28545)

**Deprecated**:
* `xformers` backend (#29262)
* `seed=None` (#29185)

**Scheduled Removals** (will be removed in future release):
* `ParallelConfig`'s direct child EPLB fields (#29324)
* `guided_*` config fields (#29326)
* `override_pooler_config` and `disable_log_requests` (#29402)
* `CompilationConfig.use_inductor` (#29323)
* Deprecated metrics (#29330)

**Other Breaking Changes**:
* PyTorch 2.9.0 upgrade requires CUDA 12.9 environment
* Mistral format auto-detection for model loading (#28659)


## New Contributors
* @jesse996 made their first contribution in https://github.com/vllm-project/vllm/pull/28846
* @Nepherpitou made their first contribution in https://github.com/vllm-project/vllm/pull/28960
* @Samoed made their first contribution in https://github.com/vllm-project/vllm/pull/27329
* @j20120307 made their first contribution in https://github.com/vllm-project/vllm/pull/28999
* @vnadathur made their first contribution in https://github.com/vllm-project/vllm/pull/26468
* @zhyajie made their first contribution in https://github.com/vllm-project/vllm/pull/28942
* @IzzyPutterman made their first contribution in https://github.com/vllm-project/vllm/pull/28896
* @rjrock-amd made their first contribution in https://github.com/vllm-project/vllm/pull/28905
* @zq1997 made their first contribution in https://github.com/vllm-project/vllm/pull/27715
* @shengliangxu made their first contribution in https://github.com/vllm-project/vllm/pull/28076
* @prashanth058 made their first contribution in https://github.com/vllm-project/vllm/pull/28972
* @qgallouedec made their first contribution in https://github.com/vllm-project/vllm/pull/28820
* @zhanggzh made their first contribution in https://github.com/vllm-project/vllm/pull/19347
* @pandalee99 made their first contribution in https://github.com/vllm-project/vllm/pull/26628
* @dsuhinin made their first contribution in https://github.com/vllm-project/vllm/pull/29100
* @xli made their first contribution in https://github.com/vllm-project/vllm/pull/29124
* @jeremyteboul made their first contribution in https://github.com/vllm-project/vllm/pull/29059
* @soodoshll made their first contribution in https://github.com/vllm-project/vllm/pull/28875
* @bhagyashrigai made their first contribution in https://github.com/vllm-project/vllm/pull/28957
* @skaraban3807 made their first contribution in https://github.com/vllm-project/vllm/pull/25559
* @Victor49152 made their first contribution in https://github.com/vllm-project/vllm/pull/28892
* @rjrock made their first contribution in https://github.com/vllm-project/vllm/pull/29205
* @FlintyLemming made their first contribution in https://github.com/vllm-project/vllm/pull/29182
* @madskildegaard made their first contribution in https://github.com/vllm-project/vllm/pull/29175
* @nandan2003 made their first contribution in https://github.com/vllm-project/vllm/pull/29189
* @michaelact made their first contribution in https://github.com/vllm-project/vllm/pull/29173
* @yongming-qin made their first contribution in https://github.com/vllm-project/vllm/pull/28958
* @joshiemoore made their first contribution in https://github.com/vllm-project/vllm/pull/29249
* @lim4349 made their first contribution in https://github.com/vllm-project/vllm/pull/29068
* @apinge made their first contribution in https://github.com/vllm-project/vllm/pull/28376
* @gbyu-amd made their first contribution in https://github.com/vllm-project/vllm/pull/28032
* @kflu made their first contribution in https://github.com/vllm-project/vllm/pull/29364
* @Inokinoki made their first contribution in https://github.com/vllm-project/vllm/pull/29200
* @GOavi101 made their first contribution in https://github.com/vllm-project/vllm/pull/29313
* @sts07142 made their first contribution in https://github.com/vllm-project/vllm/pull/29137
* @ivanium made their first contribution in https://github.com/vllm-project/vllm/pull/29143
* @geodavic made their first contribution in https://github.com/vllm-project/vllm/pull/28795
* @Yejing-Lai made their first contribution in https://github.com/vllm-project/vllm/pull/29473
* @Adityayxt made their first contribution in https://github.com/vllm-project/vllm/pull/29491
* @guodongxiaren made their first contribution in https://github.com/vllm-project/vllm/pull/29620
* @askliar made their first contribution in https://github.com/vllm-project/vllm/pull/29426
* @scydas made their first contribution in https://github.com/vllm-project/vllm/pull/29589
* @EanWang211123 made their first contribution in https://github.com/vllm-project/vllm/pull/29594
* @qGentry made their first contribution in https://github.com/vllm-project/vllm/pull/29506
* @HappyAmazonian made their first contribution in https://github.com/vllm-project/vllm/pull/29335
* @rgommers made their first contribution in https://github.com/vllm-project/vllm/pull/29241
* @staugust made their first contribution in https://github.com/vllm-project/vllm/pull/28840
* @mertunsall made their first contribution in https://github.com/vllm-project/vllm/pull/29667
* @dublc made their first contribution in https://github.com/vllm-project/vllm/pull/29728
* @nwaughachukwuma made their first contribution in https://github.com/vllm-project/vllm/pull/29671
* @BowTen made their first contribution in https://github.com/vllm-project/vllm/pull/29731
* @omera-nv made their first contribution in https://github.com/vllm-project/vllm/pull/29004
* @zhangruoxu made their first contribution in https://github.com/vllm-project/vllm/pull/29568
* @KKKZOZ made their first contribution in https://github.com/vllm-project/vllm/pull/29783
* @FredericOdermatt made their first contribution in https://github.com/vllm-project/vllm/pull/29784
* @Abdennacer-Badaoui made their first contribution in https://github.com/vllm-project/vllm/pull/29782
* @knlnguyen1802 made their first contribution in https://github.com/vllm-project/vllm/pull/28525
* @finbarrtimbers made their first contribution in https://github.com/vllm-project/vllm/pull/29796
* @hholtmann made their first contribution in https://github.com/vllm-project/vllm/pull/29711

**Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.11.1...v0.12.0