v0.8.5

This release contains 310 commits from 143 contributors (55 new contributors!). 

## Highlights
This release features important multi-modal bug fixes, day 0 support for Qwen3, and xgrammar's structure tag feature for tool calling.

### Model Support
* Day 0 support for Qwen3 and Qwen3MoE. This release fixes fp8 weight loading (#17318) and adds tuned MoE configs (#17328).
* Add ModernBERT (#16648)
* Add Granite Speech Support (#16246)
* Add PLaMo2 (#14323)
* Add Kimi-VL model support (#16387)
* Add Qwen2.5-Omni model support (thinker only) (#15130)
* Snowflake Arctic Embed (Family)  (#16649)
* Accuracy fixes for Llama4 Int4 (#16801), chat template for Llama 4 models (#16428), enhanced AMD support (#16674, #16847)

### V1 Engine
* Add `structural_tag` support using xgrammar (#17085)
* Disaggregated serving:
	* KV Connector API V1 (#15960)
	* Adding LMCache KV connector for v1 (#16625)
* Clean up: Remove Sampler from Model Code (#17084)
* MLA: Simplification to batch P/D reordering (#16673)
* Move usage stats to worker and start logging TPU hardware (#16211)
* Support FlashInfer Attention (#16684)
* Faster incremental detokenization (#15137)
* EAGLE-3 Support (#16937)

### Features
* Validate urls object for multimodal content parts (#16990)
* Prototype support sequence parallelism using compilation pass (#16155)
* Add sampling params to `v1/audio/transcriptions` endpoint (#16591)
* Enable vLLM to Dynamically Load LoRA from a Remote Server (#10546)
* Add `vllm bench [latency, throughput]` CLI commands (#16508)

### Performance
* Attention: 
	* FA3 decode perf improvement - single mma warp group support for head dim 128 (#16864)
	* Update to lastest FA3 code (#13111)
	* Support Cutlass MLA for Blackwell GPUs (#16032)
* MoE:
	* Add expert_map support to Cutlass FP8 MOE (#16861)
	* Add fp8_w8a8 fused MoE kernel tuning configs for DeepSeek V3/R1 on NVIDIA H20 (#16753)
* Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS (#6036)
* Optimize rotary_emb implementation to use Triton operator for improved performance (#16457)

### Hardwares
* TPU: 
	* Enable structured decoding on TPU V1 (#16499)
	* Capture multimodal encoder during model compilation (#15051)
	* Enable Top-P (#16843)
* AMD:
	* AITER Fused MOE V1 Support (#16752)
	* Integrate Paged Attention Kernel from AITER (#15001)
	* Support AITER MLA (#15893)
	* Upstream prefix prefill speed up for vLLM V1 (#13305)
	* Adding fp8 and variable length sequence support to Triton FAv2 kernel (#12591)
	* Add skinny gemms for unquantized linear on ROCm (#15830)
	* Follow-ups for Skinny Gemms on ROCm. (#17011)

### Documentation
* Add open-webui example (#16747)
* Document Matryoshka Representation Learning support (#16770)
* Add a security guide (#17230)
* Add example to run DeepSeek with Ray Serve LLM (#17134)
* Benchmarks for audio models (#16505)

### Security and Dependency Updates
* Don't bind tcp zmq socket to all interfaces (#17197)
* Use safe serialization and fix zmq setup for mooncake pipe (#17192)
* Bump Transformers to 4.51.3 (#17116)

### Build and testing
* Add property-based testing for vLLM endpoints using an API defined by an OpenAPI 3.1 schema (#16721)

### Breaking changes 🚨
* `--enable-chunked-prefill`, `--multi-step-stream-outputs`, `--disable-chunked-mm-input` can no longer explicitly be set to `False`. Instead, add `no-` to the start of the argument (i.e. `--enable-chunked-prefill` and `--no-enable-chunked-prefill`) (https://github.com/vllm-project/vllm/pull/16533)

## What's Changed
* Improve configs - `SchedulerConfig` by @hmellor in https://github.com/vllm-project/vllm/pull/16533
* [Misc] remove warning if triton>=3.2.0 by @DefTruth in https://github.com/vllm-project/vllm/pull/16553
* [Misc] refactor examples by @reidliu41 in https://github.com/vllm-project/vllm/pull/16563
* [Misc] Update usage with mooncake lib for kv transfer by @ShangmingCai in https://github.com/vllm-project/vllm/pull/16523
* [fix]: Dockerfile.ppc64le fixes for opencv-python and hf-xet by @Shafi-Hussain in https://github.com/vllm-project/vllm/pull/16048
* [Bugfix] Multi-modal caches not acting like LRU caches by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/16593
* [TPU][V1] Fix exponential padding when `max-num-batched-tokens` is not a power of 2 by @NickLucche in https://github.com/vllm-project/vllm/pull/16596
* Fix triton install condition on CPU by @hmellor in https://github.com/vllm-project/vllm/pull/16600
* s390x: Fix PyArrow build and add CPU test script for Buildkite CI by @Nash-123 in https://github.com/vllm-project/vllm/pull/16036
* [Model][VLM] Add Kimi-VL model support by @courage17340 in https://github.com/vllm-project/vllm/pull/16387
* [Hardware][TPU] Add torchvision to tpu dependency file by @lsy323 in https://github.com/vllm-project/vllm/pull/16616
* [DOC][TPU] Add core idea about avoiding recompilation after warmup by @yaochengji in https://github.com/vllm-project/vllm/pull/16614
* config check sleep mode support oot platforms by @celestialli in https://github.com/vllm-project/vllm/pull/16562
* [Core][Bugfix] Fix Offline MM Beam Search by @alex-jw-brooks in https://github.com/vllm-project/vllm/pull/16390
* [Kernel] moe wna16 marlin kernel by @jinzhen-lin in https://github.com/vllm-project/vllm/pull/14447
* [BugFix]: Update minimum `pyzmq` version by @taneem-ibrahim in https://github.com/vllm-project/vllm/pull/16549
* [Bugfix] Fix tests/kernels/test_mamba_ssm_ssd.py by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/16623
* [Bugfix] Fix broken GritLM model and tests (missing pooling_metadata) by @pooyadavoodi in https://github.com/vllm-project/vllm/pull/16631
* Add `vllm bench [latency, throughput]` CLI commands by @mgoin in https://github.com/vllm-project/vllm/pull/16508
* Fix vLLM x torch.compile config caching by @zou3519 in https://github.com/vllm-project/vllm/pull/16491
* [Misc] refactor argument parsing in examples by @reidliu41 in https://github.com/vllm-project/vllm/pull/16635
* [CI/Build] Fix LoRA OOM by @jeejeelee in https://github.com/vllm-project/vllm/pull/16624
* Add "/server_info" endpoint in api_server to retrieve the vllm_config.  by @Cangxihui in https://github.com/vllm-project/vllm/pull/16572
* [Kernel] Remove redundant Exp calculations by @DefTruth in https://github.com/vllm-project/vllm/pull/16123
* [Misc] Update `compressed-tensors` WNA16 to support zero-points by @dsikka in https://github.com/vllm-project/vllm/pull/14211
* [Misc] Enable vLLM to Dynamically Load LoRA from a Remote Server by @angkywilliam in https://github.com/vllm-project/vllm/pull/10546
* [Model] Add PLaMo2 by @Alnusjaponica in https://github.com/vllm-project/vllm/pull/14323
* [Bugfix] fix gpu docker image mis benchmarks dir by @lengrongfu in https://github.com/vllm-project/vllm/pull/16628
* [Misc] Modify LRUCache touch by @jeejeelee in https://github.com/vllm-project/vllm/pull/16689
* Disable remote caching when calling compile_fx by @zou3519 in https://github.com/vllm-project/vllm/pull/16611
* [Feature] add model aware kv ops helper by @billishyahao in https://github.com/vllm-project/vllm/pull/16020
* [ROCM] Bind triton version to 3.2 in requirements-built.txt  by @SageMoore in https://github.com/vllm-project/vllm/pull/16664
* [V1][Structured Output] Move xgrammar related utils to `backend_xgrammar.py` by @shen-shanshan in https://github.com/vllm-project/vllm/pull/16578
* [CI] Cleanup `additional_dependencies: [toml]` for pre-commit yapf hook by @yankay in https://github.com/vllm-project/vllm/pull/16405
* [Misc] refactor examples series by @reidliu41 in https://github.com/vllm-project/vllm/pull/16708
* [Doc] Improve OOM troubleshooting by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/16704
* [Bugfix][Kernel] fix potential cuda graph broken for merge_attn_states kernel by @DefTruth in https://github.com/vllm-project/vllm/pull/16693
* [Model] support modernbert  by @xsank in https://github.com/vllm-project/vllm/pull/16648
* [Hardware] Add processor inputs to platform validation by @joerunde in https://github.com/vllm-project/vllm/pull/16680
* Improve error for structured output backend selection by @hmellor in https://github.com/vllm-project/vllm/pull/16717
* [Misc] Remove redundant comment by @jianzs in https://github.com/vllm-project/vllm/pull/16703
* Help user create custom model for Transformers backend remote code models by @hmellor in https://github.com/vllm-project/vllm/pull/16719
* [V1][Performance] Implement custom serializaton for MultiModalKwargs [Rebased] by @p88h in https://github.com/vllm-project/vllm/pull/16432
* [V1][Spec Dec Bug Fix] Respect Spec Dec Method Specification by @luyuzhe111 in https://github.com/vllm-project/vllm/pull/16636
* Adding vllm buildkite job for IBM Power by @AaruniAggarwal in https://github.com/vllm-project/vllm/pull/16679
* [V1][Frontend] Improve Shutdown And Logs by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11737
* [rocm][V0] fix selection logic for custom PA in V0 by @divakar-amd in https://github.com/vllm-project/vllm/pull/16426
* [Bugfix] Update Florence-2 tokenizer to make grounding tasks work by @Isotr0py in https://github.com/vllm-project/vllm/pull/16734
* [Bugfix] Revert max_prompt_len validation for decoder-only models. by @davidheineman in https://github.com/vllm-project/vllm/pull/16741
* [V1] Remove log noise when idle by @russellb in https://github.com/vllm-project/vllm/pull/16735
* [Ray] Improve documentation on batch inference by @richardliaw in https://github.com/vllm-project/vllm/pull/16609
* [misc] ignore marlin_moe_wna16 local gen codes by @DefTruth in https://github.com/vllm-project/vllm/pull/16760
* [Doc] Add more tips to avoid OOM by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/16765
* [doc] add open-webui example by @reidliu41 in https://github.com/vllm-project/vllm/pull/16747
* [Bugfix] Fix GLM4 model by @intervitens in https://github.com/vllm-project/vllm/pull/16618
* [Doc] Fix a 404 link in installation/cpu.md by @windsonsea in https://github.com/vllm-project/vllm/pull/16773
* [Misc] refactor examples series - lmcache by @reidliu41 in https://github.com/vllm-project/vllm/pull/16758
* Improve configs - `TokenizerPoolConfig` + `DeviceConfig` by @hmellor in https://github.com/vllm-project/vllm/pull/16603
* fix: hyperlink by @reidliu41 in https://github.com/vllm-project/vllm/pull/16778
* [Doc] Make sure to update vLLM when installing latest code by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/16781
* [Doc] Document Matryoshka Representation Learning support by @noooop in https://github.com/vllm-project/vllm/pull/16770
* [Doc] Changed explanation of generation_tokens_total and prompt_tokens_total counter type metrics to avoid confusion by @insukim1994 in https://github.com/vllm-project/vllm/pull/16784
* [V1][Perf] Faster incremental detokenization by @njhill in https://github.com/vllm-project/vllm/pull/15137
* [Bugfix]Fix index out of range error in api server log by @WangErXiao in https://github.com/vllm-project/vllm/pull/16787
* [Kernel] Add fp8_w8a8 fused MoE kernel tuning configs for DeepSeek V3/R1 on NVIDIA H20 by @Ximingwang-09 in https://github.com/vllm-project/vllm/pull/16753
* [Model] use AutoWeightsLoader for olmoe,opt,orion,persimmon,phi3_small by @lengrongfu in https://github.com/vllm-project/vllm/pull/16548
* [TPU][V1] Fix padding recompilation when `max-num-batched-tokens` is not even by @NickLucche in https://github.com/vllm-project/vllm/pull/16726
* [V1][TPU] Enable Top K by @NickLucche in https://github.com/vllm-project/vllm/pull/15489
* [ROCM] enable aiter fused moe kernel for llama4 bf16 checkpoints by @sijiac in https://github.com/vllm-project/vllm/pull/16674
* [V1][Metrics] Fix http metrics middleware by @markmc in https://github.com/vllm-project/vllm/pull/15894
* [MLA] Simplification to batch P/D reordering by @njhill in https://github.com/vllm-project/vllm/pull/16673
* [P/D][V1] KV Connector API V1 by @ApostaC in https://github.com/vllm-project/vllm/pull/15960
* [Attention] Update to lastest FA3 code by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/13111
* Add property-based testing for vLLM endpoints using an API defined by an OpenAPI 3.1 schema by @tarukumar in https://github.com/vllm-project/vllm/pull/16721
* [Doc] Improve help examples for `--compilation-config` by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/16729
* [Misc] Update outdated note: LMCache now supports chunked prefill by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/16697
* [V1][Structured Output] Minor modification to `_validate_structured_output()` by @shen-shanshan in https://github.com/vllm-project/vllm/pull/16748
* Add hardware print to TPU V1 test by @mgoin in https://github.com/vllm-project/vllm/pull/16792
* [BugFix] Accuracy fix for llama4 int4 - improperly casted scales by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/16801
* Improve configs - `MultiModalConfig` + `PoolerConfig` + `DecodingConfig` by @hmellor in https://github.com/vllm-project/vllm/pull/16789
* [Misc] add collect_env to cli and docker image by @lengrongfu in https://github.com/vllm-project/vllm/pull/16759
* [ROCm] [Attention] Cleanup ROCm output passing by @ProExpertProg in https://github.com/vllm-project/vllm/pull/16431
* [Bugfix] fix pp for llama4 by @luccafong in https://github.com/vllm-project/vllm/pull/16746
* [Doc] add podman setup instructions for official image by @nathan-weinberg in https://github.com/vllm-project/vllm/pull/16796
* [Docs] Fix a link and grammar issue in production-stack.md by @windsonsea in https://github.com/vllm-project/vllm/pull/16809
* [Model] use AutoWeightsLoader for BigCode, GPT-J by @jonghyunchoe in https://github.com/vllm-project/vllm/pull/16823
* [Misc] Clean up Kimi-VL by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/16833
* Fix `nullable_kvs` fallback by @hmellor in https://github.com/vllm-project/vllm/pull/16837
* [New Model]: Snowflake Arctic Embed (Family)  by @noooop in https://github.com/vllm-project/vllm/pull/16649
* [Misc] refactor examples series - Chat Completion Client With Tools by @reidliu41 in https://github.com/vllm-project/vllm/pull/16829
* [Doc] Updated Llama section in tool calling docs to have llama 3.2 config info by @jmho in https://github.com/vllm-project/vllm/pull/16857
* publish neuron docker image by @omrishiv in https://github.com/vllm-project/vllm/pull/16733
* [Model][VLM] Add Qwen2.5-Omni model support (thinker only) by @fyabc in https://github.com/vllm-project/vllm/pull/15130
* [rocm][MI300] llama4 maverick fp8 moe config tp8 by @divakar-amd in https://github.com/vllm-project/vllm/pull/16847
* [Frontend] Add sampling params to `v1/audio/transcriptions` endpoint by @NickLucche in https://github.com/vllm-project/vllm/pull/16591
* [Misc] Benchmarks for audio models by @NickLucche in https://github.com/vllm-project/vllm/pull/16505
* [V1][Misc] stop update prefix cache stats when logs_stats is disabled by @vie-serendipity in https://github.com/vllm-project/vllm/pull/16460
* [Model] Refactor Phi-4-multimodal to use merged processor and support V1 by @Isotr0py in https://github.com/vllm-project/vllm/pull/15477
* [Model] Qwen2.5-Omni Cleanup  by @ywang96 in https://github.com/vllm-project/vllm/pull/16872
* [VLM] Clean up models by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/16873
* [doc] update hyperlink by @reidliu41 in https://github.com/vllm-project/vllm/pull/16877
* Log how much time loading a compiled artifact takes by @zou3519 in https://github.com/vllm-project/vllm/pull/16848
* Serialize tensors using int8 views by @p88h in https://github.com/vllm-project/vllm/pull/16866
* Improve configs - `CacheConfig` by @hmellor in https://github.com/vllm-project/vllm/pull/16835
* [easy] Pass compile_fx only the config patches by @zou3519 in https://github.com/vllm-project/vllm/pull/16845
* [Bugfix] Fix v1/spec_decode/test_ngram.py by @zixi-qi in https://github.com/vllm-project/vllm/pull/16895
* [CI/CD][V1] Add spec decode tests to CI by @WoosukKwon in https://github.com/vllm-project/vllm/pull/16900
* [Bugfix] Fix distributed bug in Qwen2.5-VL & Qwen2.5-Omni by @fyabc in https://github.com/vllm-project/vllm/pull/16907
* [Doc] Split dummy_processor_inputs() in Multimodal Docs by @alex-jw-brooks in https://github.com/vllm-project/vllm/pull/16915
* Restore buffers when wake up from level 2 sleep (#16564) by @fingertap in https://github.com/vllm-project/vllm/pull/16889
* [Misc] fix collect_env version parse by @wangxiyuan in https://github.com/vllm-project/vllm/pull/15267
* [Misc] Refactor platform to get device specific stream and event by @shen-shanshan in https://github.com/vllm-project/vllm/pull/14411
* [Bugfix] Fix GLM rotary_dim issue and support v1 by @Isotr0py in https://github.com/vllm-project/vllm/pull/16912
* Raise error for data-parallel with benchmark_throughput by @kartikx in https://github.com/vllm-project/vllm/pull/16737
* [XPU][Bugfix] minor fix for XPU by @yma11 in https://github.com/vllm-project/vllm/pull/15591
* [doc] install required python3-dev apt package by @davidxia in https://github.com/vllm-project/vllm/pull/16888
* [Doc] mention how to install in CPU editable mode by @davidxia in https://github.com/vllm-project/vllm/pull/16923
* [Core] Speed up decode by remove synchronizing operation in sampler by @chanh in https://github.com/vllm-project/vllm/pull/16436
* [V1][Spec Decode] Handle draft tokens beyond max_model_len by @WoosukKwon in https://github.com/vllm-project/vllm/pull/16087
* [TPU][V1] Implicitly adjust page size when there's SMEM OOM by @yaochengji in https://github.com/vllm-project/vllm/pull/16871
* Update Qwen1.5-MoE-W4A16-compressed-tensors.yaml by @mgoin in https://github.com/vllm-project/vllm/pull/16946
* [TPU][V1] Capture multimodal encoder during model compilation by @NickLucche in https://github.com/vllm-project/vllm/pull/15051
* [V1] V1 FlashInfer Attention by @mgoin in https://github.com/vllm-project/vllm/pull/16684
* [TPU][V1] Enable Top-P by @NickLucche in https://github.com/vllm-project/vllm/pull/16843
* [Doc] Remove unnecessary V1 flag by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/16924
* [BugFix][Spec Decode] No in-place update to draft probs by @WoosukKwon in https://github.com/vllm-project/vllm/pull/16952
* [Bugfix]: fix issue with n>1 sampling on v1 requests overriding each other by @jeffrey-dot-li in https://github.com/vllm-project/vllm/pull/16863
* [ROCm] Add aiter tkw1 kernel for Llama4 fp8 by @kliuae in https://github.com/vllm-project/vllm/pull/16727
* [Misc] Remove the chunked prefill warning for LoRA  by @jeejeelee in https://github.com/vllm-project/vllm/pull/16925
* [Kernel] Add expert_map support to Cutlass FP8 MOE by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/16861
* [V1] Remove additional_config check by @wangxiyuan in https://github.com/vllm-project/vllm/pull/16710
* [Performance][ROCm] Add skinny gemms for unquantized linear on ROCm by @charlifu in https://github.com/vllm-project/vllm/pull/15830
* Support S3 Sharded loading with RunAI Model Streamer by @omer-dayan in https://github.com/vllm-project/vllm/pull/16317
* [Bugfix] Fix f-string for Python 3.9-3.11 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/16962
* [Doc] Update ai_accelerator/hpu-gaudi.inc.md by @windsonsea in https://github.com/vllm-project/vllm/pull/16956
* [Perf] Optimize `_update_states` for GPU model runner by @SnowCharmQ in https://github.com/vllm-project/vllm/pull/16910
* [Bugfix] Fix the issue where llm.generate cannot be called repeatedly after setting GuidedDecodingParams by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/16767
* [Model] Use autoweightloader for mamba by @sfeng33 in https://github.com/vllm-project/vllm/pull/16950
* [V1] Remove pre-allocation for KV cache by @WoosukKwon in https://github.com/vllm-project/vllm/pull/16941
* [Kernel] Support Microsoft Runtime Kernel Lib for our Low Precision Computation - BitBLAS by @LeiWang1999 in https://github.com/vllm-project/vllm/pull/6036
* [BugFix] Fix incremental detokenization perf issue by @njhill in https://github.com/vllm-project/vllm/pull/16963
* [Doc] Improve documentation for multimodal CLI args by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/16960
* [FEAT][ROCm] Integrate Paged Attention Kernel from AITER by @vllmellm in https://github.com/vllm-project/vllm/pull/15001
* [Misc] refactor example series by @reidliu41 in https://github.com/vllm-project/vllm/pull/16972
* [Bugfix] Fix distributed bug again in Qwen2.5-VL & Qwen2.5-Omni by @fyabc in https://github.com/vllm-project/vllm/pull/16974
* Improve configs - `SpeculativeConfig` by @hmellor in https://github.com/vllm-project/vllm/pull/16971
* [BugFix] Pass in correct VLLM config in FlashInfer backend (#13207) by @timzsu in https://github.com/vllm-project/vllm/pull/16973
* [Misc] Add S3 environment variables for better support of MinIO. by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/16977
* [frontend] enhance tool_calls type check by @reidliu41 in https://github.com/vllm-project/vllm/pull/16882
* [FEAT][ROCm]: Support AITER MLA by @vllmellm in https://github.com/vllm-project/vllm/pull/15893
* Add assertion for no objects while hashing hf_config by @zou3519 in https://github.com/vllm-project/vllm/pull/16930
* Fencing Kernels Tests for enabling on AMD by @Alexei-V-Ivanov-AMD in https://github.com/vllm-project/vllm/pull/16929
* [BugFix] Remove default multiproc executor `collective_rpc` timeout by @njhill in https://github.com/vllm-project/vllm/pull/17000
* [Core][V1][TPU] Enable structured decoding on TPU V1 by @Chenyaaang in https://github.com/vllm-project/vllm/pull/16499
* [Bugfix] validate urls object for multimodal content parts by @gcalmettes in https://github.com/vllm-project/vllm/pull/16990
* add Dockerfile build vllm against torch nightly by @yangw-dev in https://github.com/vllm-project/vllm/pull/16936
* [Kernel][ROCM] Upstream prefix prefill speed up for vLLM V1 by @maleksan85 in https://github.com/vllm-project/vllm/pull/13305
* [V1][DP] More robust DP/EP dummy request coordination by @njhill in https://github.com/vllm-project/vllm/pull/16277
* [BugFix] Revert ROCm Custom Paged Attention Env Flag Check by @vllmellm in https://github.com/vllm-project/vllm/pull/17022
* Revert "[Misc] Add S3 environment variables for better support of MinIO." by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/17021
* [misc] tune some env vars for GB200 by @youkaichao in https://github.com/vllm-project/vllm/pull/16992
* [INTEL-HPU][v0] Port delayed sampling to upstream by @xuechendi in https://github.com/vllm-project/vllm/pull/16949
* [doc] add download path tips by @reidliu41 in https://github.com/vllm-project/vllm/pull/17013
* [Bugfix] Triton FA function takes no keyword arguments by @vllmellm in https://github.com/vllm-project/vllm/pull/16902
* [V1] Avoid socket errors during shutdown when requests are in in-flight by @njhill in https://github.com/vllm-project/vllm/pull/16807
* [BugFix] llama4 fa3 fix - RuntimeError: scheduler_metadata must have shape (metadata_size) by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/16998
* [Misc] Improve readability of get_open_port function. by @gitover22 in https://github.com/vllm-project/vllm/pull/17024
* [Bugfix] Fix AssertionError: skip_special_tokens=False is not supported for Mistral tokenizers by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/16964
* [CI] Run v1/test_serial_utils.py in CI by @russellb in https://github.com/vllm-project/vllm/pull/16996
* Mistral-format support for compressed-tensors by @mgoin in https://github.com/vllm-project/vllm/pull/16803
* Categorize `tests/kernels/` based on kernel type by @mgoin in https://github.com/vllm-project/vllm/pull/16799
* [Doc] Add top anchor and a note to quantization/bitblas.md by @windsonsea in https://github.com/vllm-project/vllm/pull/17042
* Ensure that `pid` passed to `kill_process_tree` is `int` for `mypy` by @hmellor in https://github.com/vllm-project/vllm/pull/17051
* [CI] Update structured-output label automation by @russellb in https://github.com/vllm-project/vllm/pull/17055
* Improve Transformers backend model loading QoL by @hmellor in https://github.com/vllm-project/vllm/pull/17039
* `CacheConfig.block_size` should always be `int` when used by @hmellor in https://github.com/vllm-project/vllm/pull/17052
* Use `@property` and private field for `data_parallel_rank_local` by @hmellor in https://github.com/vllm-project/vllm/pull/17053
* [Frontend] Support guidance:no-additional-properties for compatibility with xgrammar by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/15949
* [BugFix][V1] Fix int32 token index overflow when preparing input ids by @sarckk in https://github.com/vllm-project/vllm/pull/16806
* [V1][Spec Decode] Always use argmax for sampling draft tokens  by @WoosukKwon in https://github.com/vllm-project/vllm/pull/16899
* [CI/Build] workaround for CI build failure by @csy1204 in https://github.com/vllm-project/vllm/pull/17070
* [Quantization]add prefix for commandA quantized model by @CXIAAAAA in https://github.com/vllm-project/vllm/pull/17017
* [Minor] Use larger batch sizes for A100/B100/B200/MI300x by @WoosukKwon in https://github.com/vllm-project/vllm/pull/17073
* [Bugfix] Enable V1 usage stats by @mgoin in https://github.com/vllm-project/vllm/pull/16986
* More informative error when using Transformers backend by @hmellor in https://github.com/vllm-project/vllm/pull/16988
* Addendum Fix to support FIPS enabled machines with MD5 hashing by @sydarb in https://github.com/vllm-project/vllm/pull/17043
* [Bugfix][Core] add seq_id_to_seq_group clearing to avoid memory leak when s… by @zhangyuygss in https://github.com/vllm-project/vllm/pull/16472
* [V1] Update structured output by @reidliu41 in https://github.com/vllm-project/vllm/pull/16812
* [doc] update to hyperlink by @reidliu41 in https://github.com/vllm-project/vllm/pull/17096
* Add docs for runai_streamer_sharded by @omer-dayan in https://github.com/vllm-project/vllm/pull/17093
* [Chore] Remove Sampler from Model Code by @WoosukKwon in https://github.com/vllm-project/vllm/pull/17084
* Disable enforce_eager for V1 TPU sampler and structured output tests by @mgoin in https://github.com/vllm-project/vllm/pull/17016
* Simplify `TokenizerGroup` by @hmellor in https://github.com/vllm-project/vllm/pull/16790
* Fix OOT registration test by @hmellor in https://github.com/vllm-project/vllm/pull/17099
* [V1][PP] Optimization: continue scheduling prefill chunks by @ruisearch42 in https://github.com/vllm-project/vllm/pull/17080
* [Misc] Remove OLMo2 config copy by @Isotr0py in https://github.com/vllm-project/vllm/pull/17066
* Improve static type checking in `LoRAModelRunnerMixin` by @hmellor in https://github.com/vllm-project/vllm/pull/17104
* [V1][Structured Output] Clear xgrammar compiler object when engine core shut down to avoid nanobind leaked warning by @shen-shanshan in https://github.com/vllm-project/vllm/pull/16954
* [Frontend] Using matryoshka_dimensions control the allowed output dimensions. by @noooop in https://github.com/vllm-project/vllm/pull/16970
* Add missing rocm_skinny_gemms kernel test to CI by @mgoin in https://github.com/vllm-project/vllm/pull/17060
* [Misc] refactor example series - structured outputs by @reidliu41 in https://github.com/vllm-project/vllm/pull/17040
* [V1][Spec Decoding] Add num_drafts and num_accepted_tokens_per_position metrics by @markmc in https://github.com/vllm-project/vllm/pull/16665
* [CI] Add automation for the `tool-calling` github label by @russellb in https://github.com/vllm-project/vllm/pull/17118
* Updating builkite job for IBM Power  by @AaruniAggarwal in https://github.com/vllm-project/vllm/pull/17111
* existing torch installation pip command fix for docs by @atilla00 in https://github.com/vllm-project/vllm/pull/17059
* Molmo Requirements by @Eyshika in https://github.com/vllm-project/vllm/pull/17026
* Add `:markdownhelp:` to `EngineArgs` docs so markdown docstrings render properly by @hmellor in https://github.com/vllm-project/vllm/pull/17124
* Improve configs - `LoRAConfig` + `PromptAdapterConfig` by @hmellor in https://github.com/vllm-project/vllm/pull/16980
* [Docs] Generate correct github links for decorated functions by @russellb in https://github.com/vllm-project/vllm/pull/17125
* Add collective_rpc to llm engine by @yinghai in https://github.com/vllm-project/vllm/pull/16999
* Add chat template for Llama 4 models by @maxdebayser in https://github.com/vllm-project/vllm/pull/16428
* [Misc] Add example to run DeepSeek with Ray Serve LLM by @ruisearch42 in https://github.com/vllm-project/vllm/pull/17134
* Better error message for missing mistral params.json by @mgoin in https://github.com/vllm-project/vllm/pull/17132
* Use custom address for listening socket by @jglaser in https://github.com/vllm-project/vllm/pull/15988
* [FEAT] [ROCm]: AITER Fused MOE V1 Support by @vllmellm in https://github.com/vllm-project/vllm/pull/16752
* [Attention] FA3 decode perf improvement - single mma warp group support for head dim 128 by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/16864
* fix float16 support for kimi-vl by @zhouzaida in https://github.com/vllm-project/vllm/pull/17156
* [Doc] V1 : Update LoRA status by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/17133
* [Docs] Fix True->true in supported_models.md by @mgoin in https://github.com/vllm-project/vllm/pull/17141
* Move missed `SchedulerConfig` args into scheduler config group in `EngineArgs` by @hmellor in https://github.com/vllm-project/vllm/pull/17131
* [Misc] Clean up redundant code in uniproc_executor.py by @lifuhuang in https://github.com/vllm-project/vllm/pull/16762
* [Bugfix][Misc] Use TritonPlaceholderModule to defensively import triton by @MengqingCao in https://github.com/vllm-project/vllm/pull/15099
* [Misc] Benchmark Serving Script Support Appending Results by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/17028
* [Perf]Optimize rotary_emb implementation to use Triton operator for improved inference performance by @cynthieye in https://github.com/vllm-project/vllm/pull/16457
* [Bugfix] remove fallback in guided_json (int range, patterns) by @csy1204 in https://github.com/vllm-project/vllm/pull/16725
* [Quantization][FP8] Add support for FP8 models with input_scale for output projection and QK quantization by @rasmith in https://github.com/vllm-project/vllm/pull/15734
* [Doc] Add headings to improve gptqmodel.md by @windsonsea in https://github.com/vllm-project/vllm/pull/17164
* Only turn on FastIncrementalDetokenizer when tokenizers >= 0.21.1 by @houseroad in https://github.com/vllm-project/vllm/pull/17158
* [Doc] Add two links to disagg_prefill.md by @windsonsea in https://github.com/vllm-project/vllm/pull/17168
* [Doc] Move todo out of beam search docstring by @alex-jw-brooks in https://github.com/vllm-project/vllm/pull/17183
* [Bugfix] Fix mistral model tests by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/17181
* [Bugfix] Fix Mistral ChatCompletionRequest Body Exception by @JasmondL in https://github.com/vllm-project/vllm/pull/16769
* Bump Transformers to 4.51.3 by @hmellor in https://github.com/vllm-project/vllm/pull/17116
* Use Transformers helper `get_text_config()` instead of checking for `text_config` by @hmellor in https://github.com/vllm-project/vllm/pull/17105
* [doc] update wrong hf model links by @reidliu41 in https://github.com/vllm-project/vllm/pull/17184
* [Misc] Inline Molmo requirements by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/17190
* [Security] Use safe serialization and fix zmq setup for mooncake pipe by @russellb in https://github.com/vllm-project/vllm/pull/17192
* [V1] Move usage stats to worker and start logging TPU hardware by @dyli-google in https://github.com/vllm-project/vllm/pull/16211
* [Bugfix] Fix hybrid model tests by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/17182
* Fix Python packaging edge cases by @tiran in https://github.com/vllm-project/vllm/pull/17159
* [BugFix][Frontend] Fix `LLM.chat()` tokenization by @njhill in https://github.com/vllm-project/vllm/pull/16081
* [V1][Spec Decode] EAGLE-3 Support by @benchislett in https://github.com/vllm-project/vllm/pull/16937
* [Misc] Refine ray_serve_deepseek example by @ruisearch42 in https://github.com/vllm-project/vllm/pull/17204
* [Bugfix] gemma[2,3] interleaved attention when sliding window is disabled by @heheda12345 in https://github.com/vllm-project/vllm/pull/17180
* [AMD][FP8][BugFix] Remove V1 check in arg_utils.py for FP8 since it is not necessary by @rasmith in https://github.com/vllm-project/vllm/pull/17215
* [v1] [P/D] Adding LMCache KV connector for v1 by @ApostaC in https://github.com/vllm-project/vllm/pull/16625
* [Bugfix] [pytorch] Patch AOTAutogradCache._get_shape_env by @jamesjwu in https://github.com/vllm-project/vllm/pull/17142
* [MISC][AMD] Add unused annotation to rocm kernel file by @houseroad in https://github.com/vllm-project/vllm/pull/17097
* [doc] add Anything LLM integration by @reidliu41 in https://github.com/vllm-project/vllm/pull/17216
* [Minor][Spec Decode] Add use_eagle to SpeculativeConfig by @WoosukKwon in https://github.com/vllm-project/vllm/pull/17213
* [Doc] Minor fix for the vLLM TPU setup page by @yarongmu-google in https://github.com/vllm-project/vllm/pull/17206
* [Minor][Models] Fix Return Types of Llama & Eagle by @WoosukKwon in https://github.com/vllm-project/vllm/pull/17220
* Allocate kv_cache with stride order by @wenscarl in https://github.com/vllm-project/vllm/pull/16605
* [ROCm][Misc] Follow-ups for Skinny Gemms on ROCm. by @charlifu in https://github.com/vllm-project/vllm/pull/17011
* [V1][Metrics] Allow V1 AsyncLLM to use custom logger by @liuzijing2014 in https://github.com/vllm-project/vllm/pull/14661
* [BugFix] Avoid race conditions in zero-copy tensor transmission by @njhill in https://github.com/vllm-project/vllm/pull/17203
* [CI/test] Fix Eagle Correctness Test by @WoosukKwon in https://github.com/vllm-project/vllm/pull/17209
* [Core] Remove prompt string from engine core data structures by @njhill in https://github.com/vllm-project/vllm/pull/17214
* [Bugfix] Fix missing int type for `-n` in multi-image example by @Isotr0py in https://github.com/vllm-project/vllm/pull/17223
* [Bugfix] Fix standard models tests by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/17217
* [Hardware][Intel-Gaudi] Update hpu-extension and update bucketing system for HPU device by @adobrzyn in https://github.com/vllm-project/vllm/pull/17186
* [V1] Add `structural_tag` support using xgrammar by @russellb in https://github.com/vllm-project/vllm/pull/17085
* [BUGFIX] use random for NONE_HASH only when PYTHONHASHSEED not set by @andyxning in https://github.com/vllm-project/vllm/pull/17088
* [Chore] added stubs for `vllm_flash_attn` during development mode by @aarnphm in https://github.com/vllm-project/vllm/pull/17228
* [Docs] Update structured output doc for V1 by @russellb in https://github.com/vllm-project/vllm/pull/17135
* [Bugfix] fix error due to an uninitialized tokenizer when using `skip_tokenizer_init` with `num_scheduler_steps` by @junstar92 in https://github.com/vllm-project/vllm/pull/9276
* Disable the torch.compile cache checks when VLLM_DISABLE_COMPILE_CACHE=1 by @houseroad in https://github.com/vllm-project/vllm/pull/16573
* [MISC] rename interval to max_recent_requests by @andyxning in https://github.com/vllm-project/vllm/pull/14285
* [Bugfix] Fix Qwen2.5-Omni M-RoPE position ids generation by @imkero in https://github.com/vllm-project/vllm/pull/16878
* [Minor] Fix lint error in main branch by @WoosukKwon in https://github.com/vllm-project/vllm/pull/17233
* [CI/Build] remove -t for run-lm-eval-gsm-hf-baseline.sh by @reidliu41 in https://github.com/vllm-project/vllm/pull/16271
* Update test_flash_attn.py by @ShuaibinLi in https://github.com/vllm-project/vllm/pull/17102
* [Kernel][Triton][FP8] Adding fp8 and variable length sequence support to Triton FAv2 kernel by @rasmith in https://github.com/vllm-project/vllm/pull/12591
* [Misc] Make cached tokenizer pickle-compatible by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/17048
* [Bugfix] Fix QWen2 VL multimodal mapping by @jeejeelee in https://github.com/vllm-project/vllm/pull/17240
* [Bugfix] Get a specific type of layer from forward context by @heheda12345 in https://github.com/vllm-project/vllm/pull/17222
* [MISC] Use string annotation types for class definitions by @jianzs in https://github.com/vllm-project/vllm/pull/17244
* [Misc] Change buckets of histogram_iteration_tokens to [1, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8096] to represent number of tokens by @sfc-gh-zhwang in https://github.com/vllm-project/vllm/pull/17033
* [Bugfix] Fix Lora Name Parsing by @alex-jw-brooks in https://github.com/vllm-project/vllm/pull/17196
* [NVIDIA] Support Cutlass MLA for Blackwell GPUs by @kaixih in https://github.com/vllm-project/vllm/pull/16032
* [Feature] support sequence parallelism using compilation pass by @cascade812 in https://github.com/vllm-project/vllm/pull/16155
* [doc] Add feature status legend by @reidliu41 in https://github.com/vllm-project/vllm/pull/17257
* [Metrics] Fix minor inconsistencies in bucket progression by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/17262
* [V1][Spec Decode] Make eagle compatible with prefix caching. by @LiuXiaoxuanPKU in https://github.com/vllm-project/vllm/pull/17137
* [BugFix] Fix vllm_flash_attn install issues by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/17267
* [Bugfix] Fix missing ARG in Dockerfile for arm64 platforms by @lkm-schulz in https://github.com/vllm-project/vllm/pull/17261
* [Bugfix] Fix cutlass dispatch for fp8/int8 to properly invoke M<=16 c… by @Ther-LF in https://github.com/vllm-project/vllm/pull/16751
* [Bugfix] Fix Mistral3 spatial merge error by @mgoin in https://github.com/vllm-project/vllm/pull/17270
* [Doc] Fix wrong github link in LMCache examples by @KuntaiDu in https://github.com/vllm-project/vllm/pull/17274
* [Doc] small fix by @reidliu41 in https://github.com/vllm-project/vllm/pull/17277
* [Misc] Validate `stop_token_ids` contents by @njhill in https://github.com/vllm-project/vllm/pull/17268
* [Minor][Models] Pass partial_rotary_factor parameter to rope by @Eviannn in https://github.com/vllm-project/vllm/pull/17266
* [Core] Remove legacy input mapper/processor from V0 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15686
* [Model] Add Granite Speech Support by @alex-jw-brooks in https://github.com/vllm-project/vllm/pull/16246
* Update tpu_worker.py 's typo by @idouba in https://github.com/vllm-project/vllm/pull/17288
* Add missing class docstring for `PromptAdapterConfig` by @hmellor in https://github.com/vllm-project/vllm/pull/17302
* [Bugfix] Add missing `get_language_model` to new MLLMs by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/17300
* [doc] update wrong model id by @reidliu41 in https://github.com/vllm-project/vllm/pull/17287
* [Misc] Minor typo/grammar in `platforms/interface.py` by @NickLucche in https://github.com/vllm-project/vllm/pull/17307
* [Misc] Clean up Qwen2.5-Omni code by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/17301
* [Docs] Add a security guide by @russellb in https://github.com/vllm-project/vllm/pull/17230
* Improve conversion from dataclass configs to argparse arguments by @hmellor in https://github.com/vllm-project/vllm/pull/17303
* Make name of `compressed-tensors` quant method consistent across vLLM by @hmellor in https://github.com/vllm-project/vllm/pull/17255
* Explicitly explain quant method override ordering and ensure all overrides are ordered by @hmellor in https://github.com/vllm-project/vllm/pull/17256
* [Security] Don't bind tcp zmq socket to all interfaces by @russellb in https://github.com/vllm-project/vllm/pull/17197
* [Chore] cleanup license indicators in light of SPDX by @aarnphm in https://github.com/vllm-project/vllm/pull/17259
* [BugFix] Fix cascade attention - RuntimeError: scheduler_metadata must have shape (metadata_size) by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/17283
* [Bugfix] Fix moe weight losing all extra attrs after `process_weights_after_loading`. by @charlifu in https://github.com/vllm-project/vllm/pull/16854
* [Model] Qwen3 Dense FP8 Compat Fixes by @simon-mo in https://github.com/vllm-project/vllm/pull/17318

## New Contributors
* @Nash-123 made their first contribution in https://github.com/vllm-project/vllm/pull/16036
* @celestialli made their first contribution in https://github.com/vllm-project/vllm/pull/16562
* @taneem-ibrahim made their first contribution in https://github.com/vllm-project/vllm/pull/16549
* @Cangxihui made their first contribution in https://github.com/vllm-project/vllm/pull/16572
* @angkywilliam made their first contribution in https://github.com/vllm-project/vllm/pull/10546
* @Alnusjaponica made their first contribution in https://github.com/vllm-project/vllm/pull/14323
* @xsank made their first contribution in https://github.com/vllm-project/vllm/pull/16648
* @jianzs made their first contribution in https://github.com/vllm-project/vllm/pull/16703
* @p88h made their first contribution in https://github.com/vllm-project/vllm/pull/16432
* @AaruniAggarwal made their first contribution in https://github.com/vllm-project/vllm/pull/16679
* @davidheineman made their first contribution in https://github.com/vllm-project/vllm/pull/16741
* @richardliaw made their first contribution in https://github.com/vllm-project/vllm/pull/16609
* @intervitens made their first contribution in https://github.com/vllm-project/vllm/pull/16618
* @windsonsea made their first contribution in https://github.com/vllm-project/vllm/pull/16773
* @insukim1994 made their first contribution in https://github.com/vllm-project/vllm/pull/16784
* @Ximingwang-09 made their first contribution in https://github.com/vllm-project/vllm/pull/16753
* @sijiac made their first contribution in https://github.com/vllm-project/vllm/pull/16674
* @tarukumar made their first contribution in https://github.com/vllm-project/vllm/pull/16721
* @nathan-weinberg made their first contribution in https://github.com/vllm-project/vllm/pull/16796
* @jmho made their first contribution in https://github.com/vllm-project/vllm/pull/16857
* @vie-serendipity made their first contribution in https://github.com/vllm-project/vllm/pull/16460
* @zixi-qi made their first contribution in https://github.com/vllm-project/vllm/pull/16895
* @fingertap made their first contribution in https://github.com/vllm-project/vllm/pull/16889
* @kartikx made their first contribution in https://github.com/vllm-project/vllm/pull/16737
* @davidxia made their first contribution in https://github.com/vllm-project/vllm/pull/16888
* @chanh made their first contribution in https://github.com/vllm-project/vllm/pull/16436
* @jeffrey-dot-li made their first contribution in https://github.com/vllm-project/vllm/pull/16863
* @sfeng33 made their first contribution in https://github.com/vllm-project/vllm/pull/16950
* @LeiWang1999 made their first contribution in https://github.com/vllm-project/vllm/pull/6036
* @timzsu made their first contribution in https://github.com/vllm-project/vllm/pull/16973
* @yangw-dev made their first contribution in https://github.com/vllm-project/vllm/pull/16936
* @gitover22 made their first contribution in https://github.com/vllm-project/vllm/pull/17024
* @csy1204 made their first contribution in https://github.com/vllm-project/vllm/pull/17070
* @sydarb made their first contribution in https://github.com/vllm-project/vllm/pull/17043
* @zhangyuygss made their first contribution in https://github.com/vllm-project/vllm/pull/16472
* @atilla00 made their first contribution in https://github.com/vllm-project/vllm/pull/17059
* @Eyshika made their first contribution in https://github.com/vllm-project/vllm/pull/17026
* @yinghai made their first contribution in https://github.com/vllm-project/vllm/pull/16999
* @jglaser made their first contribution in https://github.com/vllm-project/vllm/pull/15988
* @zhouzaida made their first contribution in https://github.com/vllm-project/vllm/pull/17156
* @lifuhuang made their first contribution in https://github.com/vllm-project/vllm/pull/16762
* @JasmondL made their first contribution in https://github.com/vllm-project/vllm/pull/16769
* @tiran made their first contribution in https://github.com/vllm-project/vllm/pull/17159
* @jamesjwu made their first contribution in https://github.com/vllm-project/vllm/pull/17142
* @wenscarl made their first contribution in https://github.com/vllm-project/vllm/pull/16605
* @liuzijing2014 made their first contribution in https://github.com/vllm-project/vllm/pull/14661
* @adobrzyn made their first contribution in https://github.com/vllm-project/vllm/pull/17186
* @andyxning made their first contribution in https://github.com/vllm-project/vllm/pull/17088
* @junstar92 made their first contribution in https://github.com/vllm-project/vllm/pull/9276
* @ShuaibinLi made their first contribution in https://github.com/vllm-project/vllm/pull/17102
* @cascade812 made their first contribution in https://github.com/vllm-project/vllm/pull/16155
* @lkm-schulz made their first contribution in https://github.com/vllm-project/vllm/pull/17261
* @Ther-LF made their first contribution in https://github.com/vllm-project/vllm/pull/16751
* @Eviannn made their first contribution in https://github.com/vllm-project/vllm/pull/17266
* @idouba made their first contribution in https://github.com/vllm-project/vllm/pull/17288

**Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.8.4...v0.8.5