v0.1.4
vllm-project/vllmv0.1.4Aug 25, 2023by github-actions[bot]
AI Summary
v0.1.4 significantly improves usability by shipping pre-built CUDA binaries, eliminating the need for users to compile CUDA kernels locally. The release introduces support for three new models and includes optimizations for CUDA kernels and sampling parameters.
Key Highlights
- Pre-built CUDA binaries for easier installation.
- New model support: InternLM, Qwen, and Aquila.
- Optimized CUDA kernels for paged attention and approximate GELU.
- Enhanced OpenAI API compatibility and sampling alignment.
New Features
- Support for InternLM model.
- Support for Qwen (QWen-7b) model.
- Support for Aquila model.
- Support for tokens and arrays of tokens in OpenAI completions API.
- Approximate GELU kernels implementation.
Full Release Notes
## Major changes * From now on, vLLM is published with pre-built CUDA binaries. Users don't have to compile the vLLM's CUDA kernels on their machine. * New models: InternLM, Qwen, Aquila. * Optimizing CUDA kernels for paged attention and GELU. * Many bug fixes. ## What's Changed * Fix gibberish outputs of GPT-BigCode-based models by @HermitSun in https://github.com/vllm-project/vllm/pull/676 * [OPTIMIZATION] Optimizes the single_query_cached_kv_attention kernel by @naed90 in https://github.com/vllm-project/vllm/pull/420 * add QWen-7b support by @Sanster in https://github.com/vllm-project/vllm/pull/685 * add internlm model by @gqjia in https://github.com/vllm-project/vllm/pull/528 * Check the max prompt length for the OpenAI completions API by @nicobasile in https://github.com/vllm-project/vllm/pull/472 * [Fix] unwantted bias in InternLM Model by @wangruohui in https://github.com/vllm-project/vllm/pull/740 * Supports tokens and arrays of tokens as inputs to the OpenAI completion API by @wanmok in https://github.com/vllm-project/vllm/pull/715 * Fix baichuan doc style by @UranusSeven in https://github.com/vllm-project/vllm/pull/748 * Fix typo in tokenizer.py by @eltociear in https://github.com/vllm-project/vllm/pull/750 * Align with huggingface Top K sampling by @Abraham-Xu in https://github.com/vllm-project/vllm/pull/753 * explicitly del state by @cauyxy in https://github.com/vllm-project/vllm/pull/784 * Fix typo in sampling_params.py by @wangcx18 in https://github.com/vllm-project/vllm/pull/788 * [Feature | CI] Added a github action to build wheels by @Danielkinz in https://github.com/vllm-project/vllm/pull/746 * set default coompute capability according to cuda version by @zxdvd in https://github.com/vllm-project/vllm/pull/773 * Fix mqa is false case in gpt_bigcode by @zhaoyang-star in https://github.com/vllm-project/vllm/pull/806 * Add support for aquila by @shunxing1234 in https://github.com/vllm-project/vllm/pull/663 * Update Supported Model List by @zhuohan123 in https://github.com/vllm-project/vllm/pull/825 * Fix 'GPTBigCodeForCausalLM' object has no attribute 'tensor_model_parallel_world_size' by @HermitSun in https://github.com/vllm-project/vllm/pull/827 * Add compute capability 8.9 to default targets by @WoosukKwon in https://github.com/vllm-project/vllm/pull/829 * Implement approximate GELU kernels by @WoosukKwon in https://github.com/vllm-project/vllm/pull/828 * Fix typo of Aquila in README.md by @ftgreat in https://github.com/vllm-project/vllm/pull/836 * Fix for breaking changes in xformers 0.0.21 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/834 * Clean up code by @wenjun93 in https://github.com/vllm-project/vllm/pull/844 * Set replacement=True in torch.multinomial by @WoosukKwon in https://github.com/vllm-project/vllm/pull/858 * Bump up the version to v0.1.4 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/846 ## New Contributors * @naed90 made their first contribution in https://github.com/vllm-project/vllm/pull/420 * @gqjia made their first contribution in https://github.com/vllm-project/vllm/pull/528 * @nicobasile made their first contribution in https://github.com/vllm-project/vllm/pull/472 * @wanmok made their first contribution in https://github.com/vllm-project/vllm/pull/715 * @UranusSeven made their first contribution in https://github.com/vllm-project/vllm/pull/748 * @eltociear made their first contribution in https://github.com/vllm-project/vllm/pull/750 * @Abraham-Xu made their first contribution in https://github.com/vllm-project/vllm/pull/753 * @cauyxy made their first contribution in https://github.com/vllm-project/vllm/pull/784 * @wangcx18 made their first contribution in https://github.com/vllm-project/vllm/pull/788 * @Danielkinz made their first contribution in https://github.com/vllm-project/vllm/pull/746 * @zhaoyang-star made their first contribution in https://github.com/vllm-project/vllm/pull/806 * @shunxing1234 made their first contribution in https://github.com/vllm-project/vllm/pull/663 * @ftgreat made their first contribution in https://github.com/vllm-project/vllm/pull/836 * @wenjun93 made their first contribution in https://github.com/vllm-project/vllm/pull/844 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.1.3...v0.1.4