v0.3.2
vllm-project/vllmv0.3.2Feb 21, 2024by github-actions[bot]
AI Summary
Adds support for the OLMo and Gemma models, introduces the seed parameter, and enables multi-LoRA support in the OpenAI server.
Key Highlights
- OLMo model support
- Gemma model support
- Seed parameter support
- Multi-LoRA support in OpenAI server
New Features
- OLMo support
- Gemma support
- Seed parameter
- Multi-LoRA support
Full Release Notes
## Major Changes This version adds support for the OLMo and Gemma Model, as well as `seed` parameter. ## What's Changed * Defensively copy `sampling_params` by @njhill in https://github.com/vllm-project/vllm/pull/2881 * multi-LoRA as extra models in OpenAI server by @jvmncs in https://github.com/vllm-project/vllm/pull/2775 * Add code-revision config argument for Hugging Face Hub by @mbm-ai in https://github.com/vllm-project/vllm/pull/2892 * [Minor] Small fix to make distributed init logic in worker looks cleaner by @zhuohan123 in https://github.com/vllm-project/vllm/pull/2905 * [Test] Add basic correctness test by @zhuohan123 in https://github.com/vllm-project/vllm/pull/2908 * Support OLMo models. by @Isotr0py in https://github.com/vllm-project/vllm/pull/2832 * Add warning to prevent changes to benchmark api server by @simon-mo in https://github.com/vllm-project/vllm/pull/2858 * Fix `vllm:prompt_tokens_total` metric calculation by @ronensc in https://github.com/vllm-project/vllm/pull/2869 * [ROCm] include gfx908 as supported by @jamestwhedbee in https://github.com/vllm-project/vllm/pull/2792 * [FIX] Fix beam search test by @zhuohan123 in https://github.com/vllm-project/vllm/pull/2930 * Make vLLM logging formatting optional by @Yard1 in https://github.com/vllm-project/vllm/pull/2877 * Add metrics to RequestOutput by @Yard1 in https://github.com/vllm-project/vllm/pull/2876 * Add Gemma model by @xiangxu-google in https://github.com/vllm-project/vllm/pull/2964 * Upgrade transformers to v4.38.0 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/2965 * [FIX] Add Gemma model to the doc by @zhuohan123 in https://github.com/vllm-project/vllm/pull/2966 * [ROCm] Upgrade transformers to v4.38.0 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/2967 * Support per-request seed by @njhill in https://github.com/vllm-project/vllm/pull/2514 * Bump up version to v0.3.2 by @zhuohan123 in https://github.com/vllm-project/vllm/pull/2968 ## New Contributors * @jvmncs made their first contribution in https://github.com/vllm-project/vllm/pull/2775 * @mbm-ai made their first contribution in https://github.com/vllm-project/vllm/pull/2892 * @Isotr0py made their first contribution in https://github.com/vllm-project/vllm/pull/2832 * @jamestwhedbee made their first contribution in https://github.com/vllm-project/vllm/pull/2792 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.3.1...v0.3.2