v0.8.2

This release contains important bug fix for the V1 engine's memory usage. We highly recommend you upgrading! 

## Highlights
* Revert "Use uv python for docker rather than ppa:deadsnakess/ppa (#13569)" (#15377)
* Remove openvino support in favor of external plugin (#15339)

### V1 Engine
* Fix V1 Engine crash while handling requests with duplicate request id (#15043)
* Support FP8 KV Cache (#14570, #15191)
* Add flag to disable cascade attention (#15243)
* Scheduler Refactoring: Add Scheduler Interface (#15250)
* Structured Output
	* Add `disable-any-whitespace` option support for xgrammar (#15316)
	* guidance backend for structured output + `auto` fallback mode (#14779)
* Spec Decode
	* Enable spec decode for top-p & top-k sampling (#15063)
	* Use better defaults for N-gram (#15358)
	* Update target_logits in place for rejection sampling (#15427)
* AMD
	* Enable Triton(ROCm) Attention backend for Nvidia GPUs (#14071)
* TPU
	* Support V1 Sampler for ragged attention (#14227)
	* Tensor parallel MP support (#15059)
	* MHA Pallas backend (#15288)

### Features 
* Integrate `fastsafetensors` loader for loading model weights (#10647)
* Add guidance backend for structured output (#14589)

### Others
* Add Kubernetes deployment guide with CPUs (#14865)
* Support reset prefix cache by specified device (#15003)
* Support tool calling and reasoning parser (#14511)
* Support --disable-uvicorn-access-log parameters (#14754)
* Support Tele-FLM Model (#15023)
* Add pipeline parallel support to `TransformersModel` (#12832)
* Enable CUDA graph support for llama 3.2 vision (#14917)



## What's Changed
* [FEAT]Support reset prefix cache by specified device by @maobaolong in https://github.com/vllm-project/vllm/pull/15003
* [BugFix][V1] Update stats.py by @WrRan in https://github.com/vllm-project/vllm/pull/15139
* [V1][TPU] Change kv cache shape. by @vanbasten23 in https://github.com/vllm-project/vllm/pull/15145
* [FrontEnd][Perf] `merge_async_iterators` fast-path for single-prompt requests by @njhill in https://github.com/vllm-project/vllm/pull/15150
* [Docs] Annouce Ollama and Singapore Meetups by @simon-mo in https://github.com/vllm-project/vllm/pull/15161
* [V1] TPU - Tensor parallel MP support by @alexm-redhat in https://github.com/vllm-project/vllm/pull/15059
* [BugFix] Lazily import XgrammarBackend to avoid early cuda init by @njhill in https://github.com/vllm-project/vllm/pull/15171
* [Doc] Clarify run vllm only on one node in distributed inference by @ruisearch42 in https://github.com/vllm-project/vllm/pull/15148
* Fix broken tests by @jovsa in https://github.com/vllm-project/vllm/pull/14713
* [Bugfix] Fix embedding assignment for InternVL-based models by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15086
* fix "Total generated tokens:" is 0 if using --backend tgi and --endpo… by @sywangyi in https://github.com/vllm-project/vllm/pull/14673
* [V1][TPU] Support V1 Sampler for ragged attention by @NickLucche in https://github.com/vllm-project/vllm/pull/14227
* [Benchmark] Allow oversample request in benchmark dataset by @JenZhao in https://github.com/vllm-project/vllm/pull/15170
* [Core][V0] Add guidance backend for structured output by @russellb in https://github.com/vllm-project/vllm/pull/14589
* [Doc] Update Mistral Small 3.1/Pixtral example by @ywang96 in https://github.com/vllm-project/vllm/pull/15184
* [Misc] support --disable-uvicorn-access-log parameters by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/14754
* [Attention] Flash Attention 3 - fp8 by @mickaelseznec in https://github.com/vllm-project/vllm/pull/14570
* [Doc] Update README.md by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15187
* Enable CUDA graph support for llama 3.2 vision by @mritterfigma in https://github.com/vllm-project/vllm/pull/14917
* typo: Update config.py by @WrRan in https://github.com/vllm-project/vllm/pull/15189
* [Frontend][Bugfix] support prefill decode disaggregation on deepseek by @billishyahao in https://github.com/vllm-project/vllm/pull/14824
* [release] Tag vllm-cpu with latest upon new version released by @khluu in https://github.com/vllm-project/vllm/pull/15193
* Fixing Imprecise Type Annotations by @WrRan in https://github.com/vllm-project/vllm/pull/15192
* [macOS] Ugrade pytorch to 2.6.0 by @linktohack in https://github.com/vllm-project/vllm/pull/15129
* [Bugfix] Multi-video inference on LLaVA-Onevision by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15082
* Add user forum to README by @hmellor in https://github.com/vllm-project/vllm/pull/15220
* Fix env vars for running Ray distributed backend on GKE by @richardsliu in https://github.com/vllm-project/vllm/pull/15166
* Replace `misc` issues with link to forum by @hmellor in https://github.com/vllm-project/vllm/pull/15226
* [ci] feat: make the test_torchrun_example run with tp=2, external_dp=2 by @vermouth1992 in https://github.com/vllm-project/vllm/pull/15172
* [Bugfix] fix V1 Engine crash while handling requests with duplicate request id by @JasonJ2021 in https://github.com/vllm-project/vllm/pull/15043
* [V1] Add flag to disable cascade attention by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15243
* Enforce that TP > 1 is not supported for Mamba2 if Quantization is Enabled. by @fabianlim in https://github.com/vllm-project/vllm/pull/14617
* [V1] Scheduler Refactoring [1/N] - Add Scheduler Interface by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15250
* [CI/Build] LoRA : make add_lora_test safer by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/15181
* Fix CUDA kernel index data type in vllm/csrc/quantization/fused_kernels/layernorm_utils.cuh +10 by @houseroad in https://github.com/vllm-project/vllm/pull/15159
* [Misc] Clean up the BitsAndBytes arguments by @jeejeelee in https://github.com/vllm-project/vllm/pull/15140
* [ROCM] Upgrade torch to 2.6 by @SageMoore in https://github.com/vllm-project/vllm/pull/15244
* [Bugfix] Fix incorrect qwen2.5-vl attention mask pre-computation by @Isotr0py in https://github.com/vllm-project/vllm/pull/15200
* Mention `extra_body` as a way top pass vLLM only parameters using the OpenAI client by @hmellor in https://github.com/vllm-project/vllm/pull/15240
* [V1][TPU] Speed up top-k on TPU by using torch.topk by @hyeygit in https://github.com/vllm-project/vllm/pull/15242
* [Bugfix] detect alibi and revert to FA2 by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/15231
* [Model] RE: Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies  by @cyang49 in https://github.com/vllm-project/vllm/pull/14857
* [Docs] Trim the latest news in README by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15261
* [Misc] Better RayExecutor and multiprocessing compatibility by @comaniac in https://github.com/vllm-project/vllm/pull/14705
* Add an example for reproducibility by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15262
* [Hardware][TPU] Add check for no additional graph compilation during runtime by @lsy323 in https://github.com/vllm-project/vllm/pull/14710
* [V1] Enable Triton(ROCm) Attention backend for Nvidia GPUs by @Isotr0py in https://github.com/vllm-project/vllm/pull/14071
* [Doc] Update LWS docs by @Edwinhr716 in https://github.com/vllm-project/vllm/pull/15163
* [V1] Avoid redundant input processing in n>1 case by @njhill in https://github.com/vllm-project/vllm/pull/14985
* [Feature] specify model in config.yaml  by @wayzeng in https://github.com/vllm-project/vllm/pull/14855
* [Bugfix] Add int8 torch dtype for KVCache by @shen-shanshan in https://github.com/vllm-project/vllm/pull/15260
* [Misc] Add attention mask pre-computation optimization back to Qwen2.5-VL by @Isotr0py in https://github.com/vllm-project/vllm/pull/15273
* [Bugfix] Fix incorrect resolving order for transformers fallback by @Isotr0py in https://github.com/vllm-project/vllm/pull/15279
* [V1] Fix wrong import path of get_flash_attn_version by @lhtin in https://github.com/vllm-project/vllm/pull/15280
* [Bugfix] Fix broken kernel test due to missing rename for v1 Triton backend by @Isotr0py in https://github.com/vllm-project/vllm/pull/15282
* [Misc] Add cProfile helpers by @russellb in https://github.com/vllm-project/vllm/pull/15074
* [v1] Refactor KVCacheConfig by @heheda12345 in https://github.com/vllm-project/vllm/pull/14079
* [Bugfix][VLM] fix llava processor by @MengqingCao in https://github.com/vllm-project/vllm/pull/15285
* Revert "[Feature] specify model in config.yaml  (#14855)" by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15293
* [TPU][V1] MHA Pallas backend by @NickLucche in https://github.com/vllm-project/vllm/pull/15288
* [Build/CI] Fix env var typo by @russellb in https://github.com/vllm-project/vllm/pull/15305
* [Misc] Increase RayDistributedExecutor RAY_CGRAPH_get_timeout by @ruisearch42 in https://github.com/vllm-project/vllm/pull/15301
* [Bugfix][V0] Multi-sequence logprobs streaming edge case by @andylolu2 in https://github.com/vllm-project/vllm/pull/15259
* [FEAT] [ROCm]:  Add AITER RMS Norm (Layer Norm) Feature by @tjtanaa in https://github.com/vllm-project/vllm/pull/14959
* [Doc] add load_format items in docs by @wwl2755 in https://github.com/vllm-project/vllm/pull/14804
* [Bugfix] Fix torch.compile raise FileNotFoundError by @jeejeelee in https://github.com/vllm-project/vllm/pull/15278
* [Bugfix] LoRA V0 - Fix case where `max_num_seqs` is between cudagraph capture sizes by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/15308
* [Model] Support Tele-FLM Model by @atone in https://github.com/vllm-project/vllm/pull/15023
* [V1] Add `disable-any-whitespace` option support for xgrammar by @russellb in https://github.com/vllm-project/vllm/pull/15316
* [BugFix][Typing] Fix Imprecise Type Annotations by @WrRan in https://github.com/vllm-project/vllm/pull/15208
* Remove openvino support in favor of external plugin by @russellb in https://github.com/vllm-project/vllm/pull/15339
* [doc] Add back previous news by @heheda12345 in https://github.com/vllm-project/vllm/pull/15331
* Fix v1 supported oracle for worker-cls and worker-extension-cls by @hijkzzz in https://github.com/vllm-project/vllm/pull/15324
* [V1][Usage] Refactor speculative decoding configuration and tests by @ShangmingCai in https://github.com/vllm-project/vllm/pull/14434
* [ci/build] update torch nightly version for GH200 by @youkaichao in https://github.com/vllm-project/vllm/pull/15135
* [ci/build] fix broken tests in LLM.collective_rpc by @youkaichao in https://github.com/vllm-project/vllm/pull/15350
* [Misc] Add tuned R1 w8a8 and MoE configs for NVIDIA L20 by @DefTruth in https://github.com/vllm-project/vllm/pull/15322
* [Bugfix] fix torch.compiled cache hash error by @DefTruth in https://github.com/vllm-project/vllm/pull/14953
* [V1][Spec Decode] Respect prompt_lookup_max by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15348
* [V1][Spec Decode] Use better defaults for N-gram by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15358
* [Frontend] Support tool calling and reasoning parser by @WangErXiao in https://github.com/vllm-project/vllm/pull/14511
* [Misc][Doc] Add note regarding loading `generation_config` by default by @ywang96 in https://github.com/vllm-project/vllm/pull/15281
* [V1] Enable V1 Fp8 cache for FA3 in the oracle by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/15191
* [Fix] [torch.compile] Improve UUID system for custom passes by @ProExpertProg in https://github.com/vllm-project/vllm/pull/15249
* Fix non-contiguous input passed to Marlin kernel by @Qubitium in https://github.com/vllm-project/vllm/pull/15319
* [Misc] Upgrade BNB version by @jeejeelee in https://github.com/vllm-project/vllm/pull/15183
* [Misc] Remove ignore_reinit_error for ray.init() by @ruisearch42 in https://github.com/vllm-project/vllm/pull/15373
* [Bugfix][V1] Avoid importing PreTrainedModel by @HollowMan6 in https://github.com/vllm-project/vllm/pull/15366
* [Misc] Update guided decoding logs to debug by @sfbemerk in https://github.com/vllm-project/vllm/pull/15310
* Revert "[CI/Build] Use uv python for docker rather than ppa:deadsnakess/ppa (#13569)" by @simon-mo in https://github.com/vllm-project/vllm/pull/15377
* [Kernel] allow non-contiguous input for marlin kernel by @jinzhen-lin in https://github.com/vllm-project/vllm/pull/14658
* Fix zmq IPv6 URL format error by @russellb in https://github.com/vllm-project/vllm/pull/15341
* [Bugfix] Fix chat template loading by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15143
* [distributed] fix dp group by @youkaichao in https://github.com/vllm-project/vllm/pull/15355
* [Core] Integrate `fastsafetensors` loader for loading model weights by @manish-sethi in https://github.com/vllm-project/vllm/pull/10647
* [Core] Don't force uppercase for VLLM_LOGGING_LEVEL by @russellb in https://github.com/vllm-project/vllm/pull/15306
* [V1][Minor]   fix comments by @Chen-0210 in https://github.com/vllm-project/vllm/pull/15392
* [MISC] Refine no available block debug msg by @yiliu30 in https://github.com/vllm-project/vllm/pull/15076
* [V1] Aggregate chunked prompt logprobs in model runner by @njhill in https://github.com/vllm-project/vllm/pull/14875
* [Hardware][Gaudi][Feature] Enable Dynamic MoE for Mixtral by @zhenwei-intel in https://github.com/vllm-project/vllm/pull/12303
* [DOC] Add Kubernetes deployment guide with CPUs by @terrytangyuan in https://github.com/vllm-project/vllm/pull/14865
* [Doc] Update docs on handling OOM by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15357
* [V1][Perf] Simpler request output queues by @njhill in https://github.com/vllm-project/vllm/pull/15156
* [BugFix][V1] Quick fix for min_tokens with multiple EOS by @njhill in https://github.com/vllm-project/vllm/pull/15407
* [Hardware][TPU] Skip failed compilation test by @lsy323 in https://github.com/vllm-project/vllm/pull/15421
* [Build] Cython compilation support fix by @gshtras in https://github.com/vllm-project/vllm/pull/14296
* [ROCm][Kernel] MoE weights padding by @gshtras in https://github.com/vllm-project/vllm/pull/14454
* [V1][Spec Decode] Enable spec decode for top-p & top-k sampling by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15063
* [Minor][Spec Decode] Remove compiled_softmax by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15416
* Add pipeline parallel support to `TransformersModel` by @hmellor in https://github.com/vllm-project/vllm/pull/12832
* [Misc] Remove LoRA log by @jeejeelee in https://github.com/vllm-project/vllm/pull/15388
* Revert "Fix non-contiguous input passed to Marlin kernel (#15319)" by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/15398
* [Bugfix] Fixed the issue of not being able to input video and image simultaneously by @chaunceyjiang in https://github.com/vllm-project/vllm/pull/15387
* [V1] guidance backend for structured output + `auto` fallback mode by @russellb in https://github.com/vllm-project/vllm/pull/14779
* [V1][Spec Decode] Update target_logits in place for rejection sampling by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15427

## New Contributors
* @maobaolong made their first contribution in https://github.com/vllm-project/vllm/pull/15003
* @jovsa made their first contribution in https://github.com/vllm-project/vllm/pull/14713
* @mickaelseznec made their first contribution in https://github.com/vllm-project/vllm/pull/14570
* @mritterfigma made their first contribution in https://github.com/vllm-project/vllm/pull/14917
* @billishyahao made their first contribution in https://github.com/vllm-project/vllm/pull/14824
* @linktohack made their first contribution in https://github.com/vllm-project/vllm/pull/15129
* @vermouth1992 made their first contribution in https://github.com/vllm-project/vllm/pull/15172
* @JasonJ2021 made their first contribution in https://github.com/vllm-project/vllm/pull/15043
* @hyeygit made their first contribution in https://github.com/vllm-project/vllm/pull/15242
* @wayzeng made their first contribution in https://github.com/vllm-project/vllm/pull/14855
* @lhtin made their first contribution in https://github.com/vllm-project/vllm/pull/15280
* @wwl2755 made their first contribution in https://github.com/vllm-project/vllm/pull/14804
* @atone made their first contribution in https://github.com/vllm-project/vllm/pull/15023
* @hijkzzz made their first contribution in https://github.com/vllm-project/vllm/pull/15324
* @sfbemerk made their first contribution in https://github.com/vllm-project/vllm/pull/15310
* @manish-sethi made their first contribution in https://github.com/vllm-project/vllm/pull/10647
* @yiliu30 made their first contribution in https://github.com/vllm-project/vllm/pull/15076

**Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.8.1...v0.8.2