v0.6.4.post1
vllm-project/vllmv0.6.4.post1Nov 15, 2024by github-actions[bot]
AI Summary
A patch release fixing bugs related to LoRA, imports, and compatibility issues.
Key Highlights
- Fix Falcon crash
- Fix fully sharded LoRA bug
- Fix import error in tensorizer tests
- Keep compatibility for old-style classes
New Features
- Keep compatibility for `vLLMConfig` usage in out of tree models
Full Release Notes
This patch release covers bug fixes (#10347, #10349, #10348, #10352, #10363), keep compatibility for `vLLMConfig` usage in out of tree models (#10356) ## What's Changed * Add default value to avoid Falcon crash (#5363) by @wchen61 in https://github.com/vllm-project/vllm/pull/10347 * [Misc] Fix import error in tensorizer tests and cleanup some code by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/10349 * [Doc] Remove float32 choice from --lora-dtype by @xyang16 in https://github.com/vllm-project/vllm/pull/10348 * [Bugfix] Fix fully sharded LoRA bug by @jeejeelee in https://github.com/vllm-project/vllm/pull/10352 * [Misc] Fix some help info of arg_utils to improve readability by @ShangmingCai in https://github.com/vllm-project/vllm/pull/10362 * [core][misc] keep compatibility for old-style classes by @youkaichao in https://github.com/vllm-project/vllm/pull/10356 * [Bugfix] Ensure special tokens are properly filtered out for guided structured output with MistralTokenizer by @gcalmettes in https://github.com/vllm-project/vllm/pull/10363 * [Misc] Bump up test_fused_moe tolerance by @ElizaWszola in https://github.com/vllm-project/vllm/pull/10364 * [Misc] bump mistral common version by @simon-mo in https://github.com/vllm-project/vllm/pull/10367 ## New Contributors * @wchen61 made their first contribution in https://github.com/vllm-project/vllm/pull/10347 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.6.4...v0.6.4.post1