vllm Releases

Feature release adding DeepSeek V3 support, V1 engine refinements, and new model support including Granite 3.1 and JambaForSequenceClassification.

Dec 27, 2024

A major release focusing on the V1 engine refactor, `torch.compile` integration, and expanded model support including Aria, GLM-4, and OLMo.

Dec 17, 2024

A patch release fixing bugs related to LoRA, imports, and compatibility issues.

Nov 15, 2024

A major release focusing on significant progress in the V1 engine core refactor, extensive `torch.compile` support, and a wide range of new models and hardware support including Idefics3, H2OVL-Mississippi, Qwen2-Audio, and Intel Gaudi.

Nov 15, 2024

Enhances the Rustlings learning tool with search functionality, manual exercise management commands, and progress visualization.

Nov 11, 2024

Switches the rendering backend from OpenGL to CUDA to support higher resolutions and simplify installation requirements.

Oct 29, 2024

A bugfix release addressing critical issues with Pixtral models, chat API statistics, and Molmo text-only input, alongside the addition of new models like Ministral 3B/8B and VLM2Vec.

Oct 17, 2024

A major feature release introducing new models like Granite MoE and Mamba, hardware support for x86 CPU encoder-decoder, and significant architectural changes including Block Manager V2 as default and the move of beam search to the API level.

Oct 14, 2024

A performance-focused release introducing MQLLMEngine for throughput boosts, new models like Llama 3.2 and LLaVA-Onevision, and hardware support for AMD and TPU multi-step scheduling.

Sep 25, 2024

Minimal release notes: "This includes the extended context size to 8192 so should provide 5 mins conversations on moshi-mlx."

Sep 22, 2024

Sep 20, 2024

A bugfix release focused on resolving an issue where token streaming combined with stop strings resulted in incorrect final output truncation.

Sep 13, 2024

A bugfix release addressing Pixtral model issues, implementing multistep scheduling, and fixing tool use logic and multi-image support for Qwen-VL.

Sep 13, 2024

A release introducing Pixtral, Llava-Next-Video, Qwen-VL, and Qwen2-VL support, along with performance optimizations like memory optimization for awq_gemm and LoRA load/unload capabilities in the API server.

Sep 11, 2024

Adds strict linting rules and improves the user interface of the exercise list, including scroll padding and mouse interaction.

Aug 29, 2024

This release focuses on enhancing user experience by adding instant feedback, improving error handling for installation, and optimizing the exercise checking workflow through parallelization.

Aug 9, 2024

A bugfix release to correct a configuration incompatibility between vLLM and the published Meta Llama 3.1 weights.

Jul 23, 2024

This release introduces support for Meta Llama 3.1, Mistral-Nemo, and Chameleon models. It includes significant performance improvements via AWQ Marlin kernels and new vLLM CLI tools. Additional features include CPU offloading, dynamic LoRA loading, and enhancements to TPU and pipeline parallelism support.

Jul 23, 2024

Key additions include support for PaliGemma and Fuyu-8B models, soft-tuned prompts, and experimental vLLM CLI. Performance optimizations focus on FP8 layers and ZeroMQ fallback. The release also warns users about a planned future removal of beam search functionality.

Jul 15, 2024

Minimal release notes: "See the [CHANGELOG](https://github.com/rust-lang/rustlings/blob/main/CHANGELOG.md#610-2024-07-10) ✨"

Jul 10, 2024