v0.9.1

## Highlights

This release features **274 commits, from 123 contributors (27 new contributors!)**

* Progress in large scale serving
	* DP Attention + Expert Parallelism: CUDA graph support (#18724), DeepEP dispatch-combine kernel (#18434), batched/masked DeepGEMM kernel (#19111), CUTLASS MoE kernel with PPLX (#18762)
	* Heterogeneous TP (#18833), NixlConnector Enable FlashInfer backend (#19090)
	* DP: API-server scaleout with many-to-many server-engine comms (#17546), Support DP with Ray (#18779), allow AsyncLLMEngine.generate to target a specific DP rank (#19102), data parallel rank to KVEventBatch (#18925)
	* Tooling: Simplify EP kernels installation (#19412)
* RLHF workflow: Support inplace model weights loading (#18745)
* Initial full support for Hybrid Memory Allocator (#17996), support cross-layer KV sharing (#18212)
* Add FlexAttention to vLLM V1 (#16078)
* Various production hardening related to full cuda graph mode (#19171, #19106, #19321)

### Model Support
* Support Magistral (#19193), LoRA support for InternVL (#18842), minicpm eagle support (#18943), NemotronH support (#18863, #19249)
* Enable data parallel for Llama4 vision encoder (#18368)
* Add DeepSeek-R1-0528 function call chat template (#18874)

### Hardware Support & Performance Optimizations
* Add H20-3e fused MoE kernel tuning configs for DeepSeek-R1/V3 (#19205), Qwen3-235B-A22B (#19315)
* Blackwell: Add Cutlass MLA backend (#17625), Tunings for SM100 FP8 CUTLASS kernel (#18778), Use FlashInfer by default on Blackwell GPUs (#19118), Tune `scaled_fp8_quant` by increasing vectorization (#18844)
* FP4: Add compressed-tensors NVFP4 support (#18312), FP4 MoE kernel optimization (#19110)
* CPU: V1 support for the CPU backend (#16441)
* ROCm: Add AITER grouped topk for DeepSeekV2 (#18825)
* POWER: Add IBM POWER11 Support to CPU Extension Detection (#19082)
* TPU: Initial support of model parallelism with single worker using SPMD (#18011), Multi-LoRA Optimizations for the V1 TPU backend (#15655)
* Neuron: Add multi-LoRA support for Neuron. (#18284), Add Multi-Modal model support for Neuron (#18921), Support quantization on neuron (#18283)
* Platform: Make torch distributed process group extendable (#18763)

### Engine features
* Add Lora Support to Beam Search (#18346)
* Add rerank support to run_batch endpoint (#16278)
* CLI: add run batch (#18804)
* Server: custom logging (#18403), allowed_token_ids in ChatCompletionRequest (#19143)
* `LLM` API: make use_tqdm accept a callable for custom progress bars (#19357)
* perf: [KERNEL] Sampler. CUDA kernel for applying repetition penalty (#18437)

### API Deprecations
* Disallow pos-args other than `model` when initializing `LLM` (#18802)
* Remove `inputs` arg fallback in Engine classes (#18799)
* Remove fallbacks for Embeddings API (#18795)
* Remove mean pooling default for `Qwen2EmbeddingModel` (#18913)
* Require overriding `get_dummy_text` and `get_dummy_mm_data` (#18796)
* Remove metrics that were deprecated in 0.8 (#18837)

### Documentation
* Add CLI doc (#18871)
* Update SECURITY.md with link to our security guide (#18961), Add security warning to bug report template (#19365)


## What's Changed
* [CI/Build] [TPU] Fix TPU CI exit code by @CAROLZXYZXY in https://github.com/vllm-project/vllm/pull/18282
* [Neuron] Support quantization on neuron by @aws-satyajith in https://github.com/vllm-project/vllm/pull/18283
* Support datasets in `vllm bench serve` and sync with benchmark_[serving,datasets].py by @mgoin in https://github.com/vllm-project/vllm/pull/18566
* [Bugfix] Disable prefix caching by default for benchmark by @cascade812 in https://github.com/vllm-project/vllm/pull/18771
* [Build] Fixes for CMake install by @ProExpertProg in https://github.com/vllm-project/vllm/pull/18570
* [Core] Improve Tensor serialisation by @lgeiger in https://github.com/vllm-project/vllm/pull/18774
* [rocm] Fix wrong attention log by @fxmarty-amd in https://github.com/vllm-project/vllm/pull/18764
* [Bugfix] Fix nomic max_model_len by @noooop in https://github.com/vllm-project/vllm/pull/18755
* [Bugfix]: correctly propagate errors message caught at the chat_templating step to the client by @gcalmettes in https://github.com/vllm-project/vllm/pull/18769
* [V1] fix torch profiling for V1 offline scenarios by @divakar-amd in https://github.com/vllm-project/vllm/pull/18445
* [V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (2) by @RonaldBXu in https://github.com/vllm-project/vllm/pull/18781
* [Bugfix][FailingTest]Fix test_model_load_with_params.py by @rabi in https://github.com/vllm-project/vllm/pull/18758
* [Deprecation] Require overriding `get_dummy_text` and `get_dummy_mm_data` by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18796
* [Deprecation] Remove unused sync methods in `async_timeout` by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18792
* [Deprecation] Remove fallbacks for Embeddings API by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18795
* [CI] improve embed testing by @noooop in https://github.com/vllm-project/vllm/pull/18747
* Fix PiecewiseCompileInterpreter by @zou3519 in https://github.com/vllm-project/vllm/pull/17338
* [BugFix] FA2 MLA Accuracy Issue by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/18807
* [Platform][Dist] Make torch distributed process group extendable by @MengqingCao in https://github.com/vllm-project/vllm/pull/18763
* Enable Pydantic mypy checks and convert configs to Pydantic dataclasses by @hmellor in https://github.com/vllm-project/vllm/pull/17599
* [Frontend] add run batch to CLI by @reidliu41 in https://github.com/vllm-project/vllm/pull/18804
* decrement server_load on listen for disconnect by @daniel-salib in https://github.com/vllm-project/vllm/pull/18784
* [Core] Add Lora Support to Beam Search by @alex-jw-brooks in https://github.com/vllm-project/vllm/pull/18346
* [Chore] update ty configuration by @aarnphm in https://github.com/vllm-project/vllm/pull/18839
* [Misc] fix olmoe model layer for TP > 1 by @lengrongfu in https://github.com/vllm-project/vllm/pull/18828
* [V1][Metrics] Remove metrics that were deprecated in 0.8 by @markmc in https://github.com/vllm-project/vllm/pull/18837
* [Chore][Spec Decode] Update check NoneType instead of assigning variables by @aarnphm in https://github.com/vllm-project/vllm/pull/18836
* [Hardware][TPU][V1] Multi-LoRA Optimisations for the V1 TPU backend by @Akshat-Tripathi in https://github.com/vllm-project/vllm/pull/15655
* Remove checks for `None` for fields which should never be `None` by @hmellor in https://github.com/vllm-project/vllm/pull/17985
* [Core] Enable CUDA graphs for DP + All2All kernels  by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/18724
* [Bugfix][ROCm] fix the power of 2 exception from triton_unified_attention.py when running llama4 models and unit test fix by @hongxiayang in https://github.com/vllm-project/vllm/pull/18100
* Prevent the cross-encoder logic from being applied to classification tasks by @maxdebayser in https://github.com/vllm-project/vllm/pull/18838
* Add ability to use CUDAGraphs with use_inductor=False by @zou3519 in https://github.com/vllm-project/vllm/pull/17345
* [Bugfix][TPU] fix moe custom kernel import by @yaochengji in https://github.com/vllm-project/vllm/pull/18853
* [Doc][Neuron] Update documentation for Neuron by @elaineyz in https://github.com/vllm-project/vllm/pull/18868
* Skip device and quant Pydantic validation to make plugin device work by @Yikun in https://github.com/vllm-project/vllm/pull/18843
* Fixes a dead link in nightly benchmark readme by @nerdalert in https://github.com/vllm-project/vllm/pull/18856
* [Neuron] Add multi-LoRA support for Neuron. by @aws-satyajith in https://github.com/vllm-project/vllm/pull/18284
* [LoRA] Add LoRA support for InternVL  by @jeejeelee in https://github.com/vllm-project/vllm/pull/18842
* [Doc] Remove redundant spaces from compatibility_matrix.md by @windsonsea in https://github.com/vllm-project/vllm/pull/18891
* [doc] add CLI doc by @reidliu41 in https://github.com/vllm-project/vllm/pull/18871
* [Bugfix] Fix misleading information in the documentation by @jeejeelee in https://github.com/vllm-project/vllm/pull/18845
* [Misc] Replace TODO in serving transcription by @NickLucche in https://github.com/vllm-project/vllm/pull/18895
* [Bugfix] Ensure tensors are contiguous during serialisation by @lgeiger in https://github.com/vllm-project/vllm/pull/18860
* [BugFix] Update pydantic to fix error on python 3.10 by @ProExpertProg in https://github.com/vllm-project/vllm/pull/18852
* Fix an error in dummy weight loading for quantization models by @Chenyaaang in https://github.com/vllm-project/vllm/pull/18855
* [Misc][Tools][Benchmark] Add benchmark_serving supports for llama.cpp.  by @Duyi-Wang in https://github.com/vllm-project/vllm/pull/18692
* [Doc]  Fix codeblocks formatting in LoRA adapters documentation by @Zerohertz in https://github.com/vllm-project/vllm/pull/18907
* [Bugfix] Fix the failing gte embedding test by @Isotr0py in https://github.com/vllm-project/vllm/pull/18720
* [Attention][V1] Toggle for v1 attention backend by @gshtras in https://github.com/vllm-project/vllm/pull/18275
* [ROCm][V0][Attention] Revert to the previous FA triton kernel by @gshtras in https://github.com/vllm-project/vllm/pull/18226
* [Deprecation] Disallow pos-args other than `model` when initializing `LLM` by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18802
* [Misc] Remove duplicate init for self.vllm_config by @googs1025 in https://github.com/vllm-project/vllm/pull/18896
* [V1] Allocate kv_cache with stride order for V1 by @NickLucche in https://github.com/vllm-project/vllm/pull/18775
* [BugFix] Make DP work with connector-delayed new requests by @njhill in https://github.com/vllm-project/vllm/pull/18559
* [P/D] NixlConnector DP fixes by @wseaton in https://github.com/vllm-project/vllm/pull/18903
* Use standalone_compile by default in torch >= 2.8.0 by @zou3519 in https://github.com/vllm-project/vllm/pull/18846
* [TPU] remove transpose ops in moe kernel by @yaochengji in https://github.com/vllm-project/vllm/pull/18923
* [Bugfix] Fix PP default fallback behavior for V1 by @mgoin in https://github.com/vllm-project/vllm/pull/18915
* [Misc] Update type annotation for rotary embedding `base` by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18914
* [TPU][CI/CD] Clean up docker for TPU tests. by @CAROLZXYZXY in https://github.com/vllm-project/vllm/pull/18926
* improve the robustness of parsing vlms config in AutoRound by @wenhuach21 in https://github.com/vllm-project/vllm/pull/18894
* [Bugfix] Consistent ascii handling in tool parsers by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/18883
* [Model] Use AutoWeightsLoader for mamba2 by @jinyouzhi in https://github.com/vllm-project/vllm/pull/18918
* [docs] fix: fix markdown syntax by @eric-haibin-lin in https://github.com/vllm-project/vllm/pull/18927
* [ROCm] Remove unnecessary assertion of max_model_len in ROCM_AITER_MLA attention backend. by @vllmellm in https://github.com/vllm-project/vllm/pull/18938
* [Bugfix] Remove NVFP4 scales assertions to fix load_format=dummy by @mgoin in https://github.com/vllm-project/vllm/pull/18861
* [Deprecation] Remove mean pooling default for `Qwen2EmbeddingModel` by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18913
* [Misc]Fix benchmarks/README.md for speculative decoding by @rabi in https://github.com/vllm-project/vllm/pull/18897
* [doc] add mkdocs doc by @reidliu41 in https://github.com/vllm-project/vllm/pull/18930
* [Model] Use in-place adds in SigLIP by @lgeiger in https://github.com/vllm-project/vllm/pull/18922
* [Bugfix][Failing Test] Fix test_vllm_port.py by @rabi in https://github.com/vllm-project/vllm/pull/18618
* [Misc]Fix typo by @Always-Naive in https://github.com/vllm-project/vllm/pull/18947
* [Bugfix][TPU] Fix tpu model runner testcase failure by @CAROLZXYZXY in https://github.com/vllm-project/vllm/pull/18810
* [CI/Build] remove regex from build dependencies by @dtrifiro in https://github.com/vllm-project/vllm/pull/18945
* [Feature] minicpm eagle support by @huangyuxiang03 in https://github.com/vllm-project/vllm/pull/18943
* [doc] show the count for fork and watch by @reidliu41 in https://github.com/vllm-project/vllm/pull/18950
* [Docs] Update SECURITY.md with link to our security guide by @russellb in https://github.com/vllm-project/vllm/pull/18961
* Improve "failed to get the hash of the compiled graph" error by @zou3519 in https://github.com/vllm-project/vllm/pull/18956
* [Perf] API-server scaleout with many-to-many server-engine comms  by @njhill in https://github.com/vllm-project/vllm/pull/17546
* Benchmark script for fp8 vs bf16 gemm by @mgoin in https://github.com/vllm-project/vllm/pull/17126
* [VLM] Add PP support and fix GPTQ inference for Ovis models by @Isotr0py in https://github.com/vllm-project/vllm/pull/18958
* [Misc] add group_size is -1 in awq quantization by @lengrongfu in https://github.com/vllm-project/vllm/pull/18910
* Tool parser regex timeout handling by @wseaton in https://github.com/vllm-project/vllm/pull/18960
* [Docs] Correct multiprocessing design doc by @lgeiger in https://github.com/vllm-project/vllm/pull/18964
* create util function for batched arange by @yuguo68 in https://github.com/vllm-project/vllm/pull/18937
* [Frontend] Add rerank support to run_batch endpoint by @pooyadavoodi in https://github.com/vllm-project/vllm/pull/16278
* [Misc] Fix estimated max model len msg by @sarckk in https://github.com/vllm-project/vllm/pull/18966
* [Bugfix]: Fix the incompatibility issue with Structured Outputs when Thinking is disabled by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/18879
* fix security issue of logging llm output by @luccafong in https://github.com/vllm-project/vllm/pull/18980
* [Neuron] Add Multi-Modal model support for Neuron by @aws-satyajith in https://github.com/vllm-project/vllm/pull/18921
* [doc] fix the list rendering issue - security.md by @reidliu41 in https://github.com/vllm-project/vllm/pull/18982
* [BugFix] Pydantic part 2 by @ProExpertProg in https://github.com/vllm-project/vllm/pull/18911
* [FEAT][ROCm] Add AITER grouped topk for DeepSeekV2 by @vllmellm in https://github.com/vllm-project/vllm/pull/18825
* [Bugfix] Fix for issue 17396 by @frreiss in https://github.com/vllm-project/vllm/pull/18773
* [ROCm][Kernel] Add gfx950 support for skinny gemms by @charlifu in https://github.com/vllm-project/vllm/pull/18010
* [P/D] NixlConnector use cache device index for memory registration by @ptarasiewiczNV in https://github.com/vllm-project/vllm/pull/18969
* [BugFix] Fix multi-node offline data-parallel by @njhill in https://github.com/vllm-project/vllm/pull/18981
* [Misc] add return token strs for tokenize by @reidliu41 in https://github.com/vllm-project/vllm/pull/18941
* [Misc][Benchmark] Add support for CustomDataset by @ekagra-ranjan in https://github.com/vllm-project/vllm/pull/18511
* [Bugfix] Fix EAGLE3 broken logits by @benchislett in https://github.com/vllm-project/vllm/pull/18909
* [Core] Rework dtype resolution by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18751
* [LoRA] Support dynamically initialize `packed_modules_mapping` for VLM with arbitrary components by @Isotr0py in https://github.com/vllm-project/vllm/pull/18987
* [doc] small fix -  mkdocs by @reidliu41 in https://github.com/vllm-project/vllm/pull/18996
* Let max_num_batched_tokens use human_readable_int for large numbers by @mgoin in https://github.com/vllm-project/vllm/pull/18968
* [BugFix] fix data parallel construct ipv6 url addres by @lengrongfu in https://github.com/vllm-project/vllm/pull/18991
* [BugFix] Fix incorrect metrics shutdown error log message by @njhill in https://github.com/vllm-project/vllm/pull/18992
* [doc] wrong output by @reidliu41 in https://github.com/vllm-project/vllm/pull/19000
* [Misc] reuse num_tokens_across_dp of get_dp_padding to avoid unnecessary dp all reduce in set_forward_context by @izhuhaoran in https://github.com/vllm-project/vllm/pull/18935
* [Bugfix][Nixl] Fix DP Metadata Handshake by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/19008
* [Core] Support inplace model weights loading by @22quinn in https://github.com/vllm-project/vllm/pull/18745
* [doc] add pytest tips by @reidliu41 in https://github.com/vllm-project/vllm/pull/19010
* [Model] enable data parallel for Llama4 vision encoder by @jennyyyyzhen in https://github.com/vllm-project/vllm/pull/18368
* [Frontend] enable custom logging for the uvicorn server (OpenAI API server) by @fpaupier in https://github.com/vllm-project/vllm/pull/18403
* [Bugfix][Model] Attempt to fix eagle in V0. by @gshtras in https://github.com/vllm-project/vllm/pull/18978
* add an absolute path for run.sh by @calvin0327 in https://github.com/vllm-project/vllm/pull/18258
* [Hardware][TPU] Initial support of model parallelism with single worker using SPMD by @lsy323 in https://github.com/vllm-project/vllm/pull/18011
* [Doc] Remove duplicate TOCs during MkDocs migration by @Zerohertz in https://github.com/vllm-project/vllm/pull/19021
* [Bugfix][EP+DP] Use pplx-kernel internode instead of intranode by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/19034
* Adding "LoRA Test %N" to AMD production tests by @Concurrensee in https://github.com/vllm-project/vllm/pull/18929
* [CPU][CI] Re-enable the CPU CI tests by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/19046
* [ROCm][Build] Clean up the ROCm build by @gshtras in https://github.com/vllm-project/vllm/pull/19040
* [V1] Support DP with Ray by @ruisearch42 in https://github.com/vllm-project/vllm/pull/18779
* Add tarsier model support by @princepride in https://github.com/vllm-project/vllm/pull/18985
* [bugfix] small fix logic issue by @reidliu41 in https://github.com/vllm-project/vllm/pull/18999
* Reduce logs in CLI scripts and plugin loader by @mgoin in https://github.com/vllm-project/vllm/pull/18970
* [Bugfix] Use cmake 3.26.1 instead of 3.26 to avoid build failure by @houseroad in https://github.com/vllm-project/vllm/pull/19019
* [v1][KVCacheManager] Rename BlockHashType to BlockHash by @heheda12345 in https://github.com/vllm-project/vllm/pull/19015
* Update docker docs with ARM CUDA cross-compile by @mgoin in https://github.com/vllm-project/vllm/pull/19037
* [Doc] Add InternVL LoRA support  by @jeejeelee in https://github.com/vllm-project/vllm/pull/19055
* [Misc] Update `WeightsMapper` for qwen2-vl/qwen2.5-vl by @Isotr0py in https://github.com/vllm-project/vllm/pull/19054
* [Doc] Update V1 user guide for embedding and enc-dec models by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/19060
* [doc] clarify windows support by @youkaichao in https://github.com/vllm-project/vllm/pull/19088
* [CI/Build] Remove V0 LoRA test by @jeejeelee in https://github.com/vllm-project/vllm/pull/19066
* Fix underscores in dict keys passed via CLI by @hmellor in https://github.com/vllm-project/vllm/pull/19030
* [Bugfix] disable processor cache  by @zucchini-nlp in https://github.com/vllm-project/vllm/pull/19068
* [Doc] Improve the Pull Request template with key components by @houseroad in https://github.com/vllm-project/vllm/pull/19086
* [Misc] Add missing `_Backend` enums by @NickLucche in https://github.com/vllm-project/vllm/pull/19081
* [Misc] fix: add miss best_of param validation by @googs1025 in https://github.com/vllm-project/vllm/pull/18555
* [Misc] Add SPDX-FileCopyrightText  by @simon-mo in https://github.com/vllm-project/vllm/pull/19100
* [Doc] Readme standardization by @SorenDreano in https://github.com/vllm-project/vllm/pull/18695
* [doc] update docker version by @reidliu41 in https://github.com/vllm-project/vllm/pull/19074
* [Kernel] DeepEP dispatch-combine kernel integration by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/18434
* [V1] Support cross-layer KV sharing by @sarckk in https://github.com/vllm-project/vllm/pull/18212
* [Perf] Tune `scaled_fp8_quant` by increasing vectorization by @mgoin in https://github.com/vllm-project/vllm/pull/18844
* Fix interaction between `Optional` and `Annotated` in CLI typing by @hmellor in https://github.com/vllm-project/vllm/pull/19093
* [v1] Re-init input batch for multiple kv cache groups by @heheda12345 in https://github.com/vllm-project/vllm/pull/18654
* [V1][Spec Decode][Ngram] 1.35x gain -> 1.95x gain on InstructCoder with prompt fix by @ekagra-ranjan in https://github.com/vllm-project/vllm/pull/18971
* [Bugfix] get_num_blocks_to_allocate with null_block by @heheda12345 in https://github.com/vllm-project/vllm/pull/19031
* [Bugfix]: Fix the incompatibility issue with tool_choice 'required' when Thinking is enabled by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/19075
* [Bugfix][P/D] Fix Prefix Cache Bug by @NickLucche in https://github.com/vllm-project/vllm/pull/18411
* [Bugfix] Max concurrency estimation and check_enough_kv_cache_memory for models with sliding window layers by @heheda12345 in https://github.com/vllm-project/vllm/pull/19029
* feat: add data parallel rank to KVEventBatch by @PeaBrane in https://github.com/vllm-project/vllm/pull/18925
* [Misc] Fix path and python alias errors in disagg_prefill exmaples by @Jeffwan in https://github.com/vllm-project/vllm/pull/18919
* [Docs] Add developer doc about CI failures by @russellb in https://github.com/vllm-project/vllm/pull/18782
* [CPU] V1 support for the CPU backend by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/16441
* [Core] Cast multimodal input in hf processor by @lgeiger in https://github.com/vllm-project/vllm/pull/18862
* [KERNEL] Sampler. CUDA kernel for applying repetition penalty by @vadiklyutiy in https://github.com/vllm-project/vllm/pull/18437
* [Cleanup][v1]:remote guided-decoding-backend for example by @calvin0327 in https://github.com/vllm-project/vllm/pull/19059
* [NVIDIA] Add Cutlass MLA backend by @kaixih in https://github.com/vllm-project/vllm/pull/17625
* [Bugfix] Fix FA3 full cuda graph correctness by @WoosukKwon in https://github.com/vllm-project/vllm/pull/19106
* Fix #19130 by @princepride in https://github.com/vllm-project/vllm/pull/19132
* [TPU] Skip hanging tests by @lsy323 in https://github.com/vllm-project/vllm/pull/19115
* Fix ValueError: Missing value for tag key(s): model_name,engine. by @eicherseiji in https://github.com/vllm-project/vllm/pull/19113
* [Misc] Add packages for benchmark as extra dependency by @Isotr0py in https://github.com/vllm-project/vllm/pull/19089
* Improve the output precision of embedding models by @noooop in https://github.com/vllm-project/vllm/pull/19092
* [CI/Build][Bugfix] Ensure compatibility with transformers 4.52 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18678
* Add DeepSeek-R1-0528 function call chat template by @Xu-Wenqing in https://github.com/vllm-project/vllm/pull/18874
* Sm100 blockwise fp8 swap ab by @IwakuraRein in https://github.com/vllm-project/vllm/pull/18564
* [Doc] Update V1 Guide for embedding models by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/19141
* Allow AsyncLLMEngine.generate to target a specific DP rank by @jmswen in https://github.com/vllm-project/vllm/pull/19102
* [Bugfix][EP+DP] Fix internode check by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/19112
* [Perf] Tunings for SM100 FP8 CUTLASS kernel by @mgoin in https://github.com/vllm-project/vllm/pull/18778
* [TPU] Update dynamo dump file name in compilation test by @lsy323 in https://github.com/vllm-project/vllm/pull/19108
* [Bugfix] fix v1 cpu worker fails on macOS by @kebe7jun in https://github.com/vllm-project/vllm/pull/19121
* [Kernel] Integrate batched/masked deepgemm kernel by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/19111
* [Misc] refactor: simplify EngineCoreClient.make_async_mp_client in AsyncLLM by @googs1025 in https://github.com/vllm-project/vllm/pull/18817
* [P/D] Heterogeneous TP by @NickLucche in https://github.com/vllm-project/vllm/pull/18833
* [doc] small fix by @reidliu41 in https://github.com/vllm-project/vllm/pull/19167
* [Bugfix][Nixl] Fix full prefix cache hit bug by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/18632
* [Bugfix] Fix port handling in make_zmq_path by @mgoin in https://github.com/vllm-project/vllm/pull/19117
* [Torch Nightly]add missing dependency by @yangw-dev in https://github.com/vllm-project/vllm/pull/18770
* Handle non-serializable objects when dumping benchmark results by @huydhn in https://github.com/vllm-project/vllm/pull/19114
* [BugFix][Minor] Fix full cuda graph bug when max_num_seqs < 512 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/19171
* [Bugfix]: Fix the incompatibility issue with stream when Thinking is disabled by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/19135
* [Build] Annotate wheel and container path for release workflow by @simon-mo in https://github.com/vllm-project/vllm/pull/19162
* [Misc] Remove unnecessary fallback to prefill-decode attention by @vllmellm in https://github.com/vllm-project/vllm/pull/19138
* [Misc] Do not override NCCL_CUMEM_ENABLE if set explicitly by @22quinn in https://github.com/vllm-project/vllm/pull/19105
* [Frontend] improve vllm run-batch --help display by @reidliu41 in https://github.com/vllm-project/vllm/pull/19187
* [Bugfix] properly catch PIL-related errors for vision models when incorrect data urls are provided by @gcalmettes in https://github.com/vllm-project/vllm/pull/19202
* [mistral_common] Add v11 tokenizer by @patrickvonplaten in https://github.com/vllm-project/vllm/pull/19193
* Add H20-3e fused MoE kernel tuning configs for DeepSeek-R1/V3 by @Xu-Wenqing in https://github.com/vllm-project/vllm/pull/19205
* [Hardware][NVIDIA] FP4 MoE kernel optimization by @dubcyfor3 in https://github.com/vllm-project/vllm/pull/19110
* [MISC][Bugfix] Use less CPU when message queue has been empty for some time by @p12tic in https://github.com/vllm-project/vllm/pull/16226
* [P/D][NixlConnector] Enable FlashInfer backend by @NickLucche in https://github.com/vllm-project/vllm/pull/19090
* [Quantization] Skip Fp4 Test for `compressed-tensors` by @dsikka in https://github.com/vllm-project/vllm/pull/19217
* [V1] Use FlashInfer by default on Blackwell GPUs by @mgoin in https://github.com/vllm-project/vllm/pull/19118
* [Model] NemotronH support by @vegaluisjose in https://github.com/vllm-project/vllm/pull/18863
* Fix AOPerModuleConfig name changes by @jerryzh168 in https://github.com/vllm-project/vllm/pull/18869
* [Bugfix] Fix EAGLE vocab embedding construction for Llama 70B by @benchislett in https://github.com/vllm-project/vllm/pull/19033
* [v1] Hybrid Memory Allocator by @heheda12345 in https://github.com/vllm-project/vllm/pull/17996
* [TPU] update torch_xla pin by @yaochengji in https://github.com/vllm-project/vllm/pull/19231
* Support allowed_token_ids in ChatCompletionRequest by @xu-song in https://github.com/vllm-project/vllm/pull/19143
* [Chore] update CODEOWNERS by @aarnphm in https://github.com/vllm-project/vllm/pull/19247
* [v1][P/D] Fix a edge case in kv cache schedule by @KingsleyZhang123 in https://github.com/vllm-project/vllm/pull/19182
* [TPU] fix kv cache dtype in model runner by @yaochengji in https://github.com/vllm-project/vllm/pull/19244
* [Quantization] Bump compressed-tensors version; update NVFP4A16 test model by @dsikka in https://github.com/vllm-project/vllm/pull/19224
* [Docs] Improve V1 KVConnector interface documentation by @njhill in https://github.com/vllm-project/vllm/pull/19172
* Fix CompilationConfig repr by @zou3519 in https://github.com/vllm-project/vllm/pull/19091
* Unit Test for run_dp_sharded_vision_model by @cryptopic in https://github.com/vllm-project/vllm/pull/19103
* [Model] Optimize nemotron_h implementation by @jeejeelee in https://github.com/vllm-project/vllm/pull/19249
* [Core] Raise when non-multi-instance DP clients target a DP rank by @jmswen in https://github.com/vllm-project/vllm/pull/19227
* improve logits bias by @yuguo68 in https://github.com/vllm-project/vllm/pull/19041
* Fixed ppc build when it runs on non-RHEL based linux distros by @npanpaliya in https://github.com/vllm-project/vllm/pull/18422
* [BugFix] Fix MultiConnector test after HMA changes by @njhill in https://github.com/vllm-project/vllm/pull/19291
* [Bugfix][Core] Update cancellation logic in `generate()` to handle Generator exits by @Adolfo-Karim in https://github.com/vllm-project/vllm/pull/19225
* [Core] Fix abrupt request abort by @NickLucche in https://github.com/vllm-project/vllm/pull/18485
* [BugFix] Fix tpu_model_runner block_id concatenation by @njhill in https://github.com/vllm-project/vllm/pull/19228
* [Misc][Tools][Benchmark] Fix and improve auto tune script by @Chenyaaang in https://github.com/vllm-project/vllm/pull/19163
* [Build][ROCm] Update Dockerfile.rocm by @Alexei-V-Ivanov-AMD in https://github.com/vllm-project/vllm/pull/19296
* [Easy][Test] Simplify test_function_tool_use with multiple parametrizes by @houseroad in https://github.com/vllm-project/vllm/pull/19269
* [Kernel] Integrate CUTLASS MoE kernel with PPLX by @ElizaWszola in https://github.com/vllm-project/vllm/pull/18762
* [TPU][Test] Add script to run benchmark on TPU for buildkite by @QiliangCui in https://github.com/vllm-project/vllm/pull/19039
* [CI][PowerPC] Use a more appropriate way to select testcase in tests/models/language/pooling/test_embedding.py by @AaruniAggarwal in https://github.com/vllm-project/vllm/pull/19253
* Add FlexAttention to V1 by @drisspg in https://github.com/vllm-project/vllm/pull/16078
* [Misc] refactor context extension by @reidliu41 in https://github.com/vllm-project/vllm/pull/19246
* [CI/Build] Improve Llama GGUF test robustness by @Isotr0py in https://github.com/vllm-project/vllm/pull/19287
* [Nit][Benchmark]Fix example in benchmark_serving_structured_output.py by @draftbk in https://github.com/vllm-project/vllm/pull/19311
* [AMD] Update compatible packaging version by @pramenku in https://github.com/vllm-project/vllm/pull/19309
* [BugFix][V1] Fix memory profiling bug by @ProExpertProg in https://github.com/vllm-project/vllm/pull/18974
* [Bugfix]: Fix TypeError: 'float' object cannot be interpreted as an integer by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/19283
* [Bugfix] Re-enable use_cudagraph in vLLM v1 by @zou3519 in https://github.com/vllm-project/vllm/pull/19299
* [Misc] Change tests/compile to use VLLM_V1 by default by @zou3519 in https://github.com/vllm-project/vllm/pull/19302
* Add H20-3e fused MoE kernel tuning configs for Qwen3-235B-A22B by @Xu-Wenqing in https://github.com/vllm-project/vllm/pull/19315
* [Hardware][POWER] Add IBM POWER11 Support to CPU Extension Detection by @Akashcodes732 in https://github.com/vllm-project/vllm/pull/19082
* [Quantization] Add compressed-tensors NVFP4 support by @dsikka in https://github.com/vllm-project/vllm/pull/18312
* [Multi Modal] Add an env var for message queue max chunk bytes  by @jennyyyyzhen in https://github.com/vllm-project/vllm/pull/19242
* [Bugfix] model_max_length should consider max_model_len in tokenizer_config by @noooop in https://github.com/vllm-project/vllm/pull/19201
* [Deprecation] Remove `inputs` arg fallback in Engine classes by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/18799
* [Misc] Add documentation update reminder to PR template by @Isotr0py in https://github.com/vllm-project/vllm/pull/19289
* [Frontend] Remove unreachable code from llm.py by @KsuParkhamchuk in https://github.com/vllm-project/vllm/pull/19288
* [Misc] Cleanup compilation tests by @zou3519 in https://github.com/vllm-project/vllm/pull/19343
* [doc] improve ci doc by @reidliu41 in https://github.com/vllm-project/vllm/pull/19307
* [Doc] Fix description in the Automatic Prefix Caching design doc by @cr7258 in https://github.com/vllm-project/vllm/pull/19333
* [CI/Build] Fix LoRA test by @jeejeelee in https://github.com/vllm-project/vllm/pull/19350
* [Fix] Allow kernel compilation for CUDA capability 8.7 by @conroy-cheers in https://github.com/vllm-project/vllm/pull/19328
* [CI] Introduce rules for llama auto-label by @houseroad in https://github.com/vllm-project/vllm/pull/19323
* [Docs] Fix a bullet list in usage/security.md by @windsonsea in https://github.com/vllm-project/vllm/pull/19358
* [full_graph] Fix query_start_loc padding by @yinghai in https://github.com/vllm-project/vllm/pull/19321
* [v1] Add fp32 support to v1 engine through flex attn by @Isotr0py in https://github.com/vllm-project/vllm/pull/19319
* [Misc] Fixes and Optimizations for DeepEP + DeepGEMM combination. by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/19298
* [Bugfix][Core] Prevent token lengths exceeding `max_model_len` in V0 by @22quinn in https://github.com/vllm-project/vllm/pull/19348
* [Quantization] Bump compressed-tensors version by @kylesayrs in https://github.com/vllm-project/vllm/pull/19295
* [Frontend] Make TIMEOUT_KEEP_ALIVE configurable through env var by @liusiqian-tal in https://github.com/vllm-project/vllm/pull/18472
* [TPU]Fix KV cache sharing tests by @lsy323 in https://github.com/vllm-project/vllm/pull/19371
* [HOT-FIX] Add `kv_sharing_target_layer_name` argument to cutlass_mla backend by @pavanimajety in https://github.com/vllm-project/vllm/pull/19374
* [Misc] Fix a config typo in disable_hybrid_kv_cache_manager configuration by @lsy323 in https://github.com/vllm-project/vllm/pull/19383
* [V1] Reuse V0's memory_profiling util for gpu worker memory profiling by @yeqcharlotte in https://github.com/vllm-project/vllm/pull/19312
* [Bugfix] Fix benchmark_moe.py by @gty111 in https://github.com/vllm-project/vllm/pull/19016
* Use xla flag to improve the quantized model performance by @vanbasten23 in https://github.com/vllm-project/vllm/pull/19303
* Fix docs/mkdocs/hooks/remove_announcement.py by @hmellor in https://github.com/vllm-project/vllm/pull/19382
* [Frontend] Make use_tqdm accept a callable for custom progress bars by @reidliu41 in https://github.com/vllm-project/vllm/pull/19357
* [Core] Use tuple for kv cache group block ids by @njhill in https://github.com/vllm-project/vllm/pull/19175
* [Bugfix] Fix modelscope token passed in by @Potabk in https://github.com/vllm-project/vllm/pull/19389
* [Core] Batch multi modal input using pinned memory by @lgeiger in https://github.com/vllm-project/vllm/pull/19169
* Add security warning to bug report template by @russellb in https://github.com/vllm-project/vllm/pull/19365
* [Misc] refactor neuron_multimodal and profiling by @reidliu41 in https://github.com/vllm-project/vllm/pull/19397
* Add clear documentation around the impact of debugging flag by @annapendleton in https://github.com/vllm-project/vllm/pull/19369
* Automatically bind CPU OMP Threads of a rank to CPU ids of a NUMA node. by @louie-tsai in https://github.com/vllm-project/vllm/pull/17930
* Revert "[v1] Add fp32 support to v1 engine through flex attn" by @Isotr0py in https://github.com/vllm-project/vllm/pull/19404
* [BugFix][FlashInfer] Fix attention backend interface mismatch with unexpected keyword `use_irope` by @YUNQIUGUO in https://github.com/vllm-project/vllm/pull/19134
* [BugFix][CPU] Fix CPU CI by ignore collecting test_pixtral by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/19411
* Simplify ep kernels installation by @youkaichao in https://github.com/vllm-project/vllm/pull/19412
* [Misc] Slight improvement of the BNB  by @jeejeelee in https://github.com/vllm-project/vllm/pull/19418

## New Contributors
* @nerdalert made their first contribution in https://github.com/vllm-project/vllm/pull/18856
* @Duyi-Wang made their first contribution in https://github.com/vllm-project/vllm/pull/18692
* @jinyouzhi made their first contribution in https://github.com/vllm-project/vllm/pull/18918
* @eric-haibin-lin made their first contribution in https://github.com/vllm-project/vllm/pull/18927
* @Always-Naive made their first contribution in https://github.com/vllm-project/vllm/pull/18947
* @yuguo68 made their first contribution in https://github.com/vllm-project/vllm/pull/18937
* @ptarasiewiczNV made their first contribution in https://github.com/vllm-project/vllm/pull/18969
* @izhuhaoran made their first contribution in https://github.com/vllm-project/vllm/pull/18935
* @jennyyyyzhen made their first contribution in https://github.com/vllm-project/vllm/pull/18368
* @zucchini-nlp made their first contribution in https://github.com/vllm-project/vllm/pull/19068
* @SorenDreano made their first contribution in https://github.com/vllm-project/vllm/pull/18695
* @PeaBrane made their first contribution in https://github.com/vllm-project/vllm/pull/18925
* @jmswen made their first contribution in https://github.com/vllm-project/vllm/pull/19102
* @dubcyfor3 made their first contribution in https://github.com/vllm-project/vllm/pull/19110
* @p12tic made their first contribution in https://github.com/vllm-project/vllm/pull/16226
* @KingsleyZhang123 made their first contribution in https://github.com/vllm-project/vllm/pull/19182
* @cryptopic made their first contribution in https://github.com/vllm-project/vllm/pull/19103
* @Adolfo-Karim made their first contribution in https://github.com/vllm-project/vllm/pull/19225
* @QiliangCui made their first contribution in https://github.com/vllm-project/vllm/pull/19039
* @draftbk made their first contribution in https://github.com/vllm-project/vllm/pull/19311
* @pramenku made their first contribution in https://github.com/vllm-project/vllm/pull/19309
* @KsuParkhamchuk made their first contribution in https://github.com/vllm-project/vllm/pull/19288
* @cr7258 made their first contribution in https://github.com/vllm-project/vllm/pull/19333
* @liusiqian-tal made their first contribution in https://github.com/vllm-project/vllm/pull/18472
* @annapendleton made their first contribution in https://github.com/vllm-project/vllm/pull/19369
* @louie-tsai made their first contribution in https://github.com/vllm-project/vllm/pull/17930
* @YUNQIUGUO made their first contribution in https://github.com/vllm-project/vllm/pull/19134

**Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.9.0...v0.9.1