v0.6.6.post1
vllm-project/vllmv0.6.6.post1Dec 27, 2024by github-actions[bot]
AI Summary
A patch release that restores quantization support for other MoE methods which was broken by the initial DeepSeek V3 support.
Key Highlights
- Restore functionalities for other quantized MoEs
- V1: Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling
New Features
- Document Deepseek V3 support
Full Release Notes
This release restore functionalities for other quantized MoEs, which was introduced as part of initial DeepSeek V3 support 🙇 . ## What's Changed * [Docs] Document Deepseek V3 support by @simon-mo in https://github.com/vllm-project/vllm/pull/11535 * Update openai_compatible_server.md by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/11536 * [V1] Use FlashInfer Sampling Kernel for Top-P & Top-K Sampling by @WoosukKwon in https://github.com/vllm-project/vllm/pull/11394 * [V1] Fix yapf by @WoosukKwon in https://github.com/vllm-project/vllm/pull/11538 * [CI] Fix broken CI by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/11543 * [misc] fix typing by @youkaichao in https://github.com/vllm-project/vllm/pull/11540 * [V1][3/N] API Server: Reduce Task Switching + Handle Abort Properly by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/11534 * [BugFix] Deepseekv3 broke quantization for all other methods by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/11547 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.6.6...v0.6.6.post1