v0.1.5

vllm-project/vllmv0.1.5Sep 7, 2023by github-actions[bot]

AI Summary

This release stabilizes the AsyncLLMEngine with a background loop, aligns beam search behavior with HuggingFace, and adds support for CodeLLaMA.

Key Highlights

  • Align beam search with hf_model.generate
  • Stablelize AsyncLLMEngine with a background engine loop
  • Add support for CodeLLaMA

New Features

  • Enable safetensors loading for all models
  • Fix NaN errors in paged attention kernel
  • Implement RoPE for GPT-J
  • Avoid compiling kernels for double data type

Full Release Notes

## Major Changes
* Align beam search with `hf_model.generate`.
* Stablelize AsyncLLMEngine with a background engine loop.
* Add support for CodeLLaMA.
* Add many model correctness tests.
* Many other correctness fixes.

## What's Changed
* Add support for CodeLlama by @Yard1 in https://github.com/vllm-project/vllm/pull/854
* [Fix] Fix a condition for ignored sequences by @zhuohan123 in https://github.com/vllm-project/vllm/pull/867
* use flash-attn via xformers by @tmm1 in https://github.com/vllm-project/vllm/pull/877
* Enable request body OpenAPI spec for OpenAI endpoints by @Peilun-Li in https://github.com/vllm-project/vllm/pull/865
* Accelerate LLaMA model loading by @JF-D in https://github.com/vllm-project/vllm/pull/234
* Improve _prune_hidden_states micro-benchmark by @tmm1 in https://github.com/vllm-project/vllm/pull/707
* fix: bug fix when penalties are negative by @pfldy2850 in https://github.com/vllm-project/vllm/pull/913
* [Docs] Minor fixes in supported models by @WoosukKwon in https://github.com/vllm-project/vllm/pull/920
* Fix README.md Link by @zhuohan123 in https://github.com/vllm-project/vllm/pull/927
* Add tests for models by @WoosukKwon in https://github.com/vllm-project/vllm/pull/922
* Avoid compiling kernels for double data type by @WoosukKwon in https://github.com/vllm-project/vllm/pull/933
* [BugFix] Fix NaN errors in paged attention kernel by @WoosukKwon in https://github.com/vllm-project/vllm/pull/936
* Refactor AsyncLLMEngine by @Yard1 in https://github.com/vllm-project/vllm/pull/880
* Only emit warning about internal tokenizer if it isn't being used by @nelson-liu in https://github.com/vllm-project/vllm/pull/939
* Align vLLM's beam search implementation with HF generate by @zhuohan123 in https://github.com/vllm-project/vllm/pull/857
* Initialize AsyncLLMEngine bg loop correctly by @Yard1 in https://github.com/vllm-project/vllm/pull/943
* FIx vLLM cannot launch by @HermitSun in https://github.com/vllm-project/vllm/pull/948
* Clean up kernel unit tests by @WoosukKwon in https://github.com/vllm-project/vllm/pull/938
* Use queue for finished requests by @Yard1 in https://github.com/vllm-project/vllm/pull/957
* [BugFix] Implement RoPE for GPT-J by @WoosukKwon in https://github.com/vllm-project/vllm/pull/941
* Set torch default dtype in a context manager by @Yard1 in https://github.com/vllm-project/vllm/pull/971
* Bump up transformers version in requirements.txt by @WoosukKwon in https://github.com/vllm-project/vllm/pull/976
* Make `AsyncLLMEngine` more robust & fix batched abort by @Yard1 in https://github.com/vllm-project/vllm/pull/969
* Enable safetensors loading for all models by @zhuohan123 in https://github.com/vllm-project/vllm/pull/974
* [FIX] Fix Alibi implementation in PagedAttention kernel by @zhuohan123 in https://github.com/vllm-project/vllm/pull/945
* Bump up the version to v0.1.5 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/944

## New Contributors
* @tmm1 made their first contribution in https://github.com/vllm-project/vllm/pull/877
* @Peilun-Li made their first contribution in https://github.com/vllm-project/vllm/pull/865
* @JF-D made their first contribution in https://github.com/vllm-project/vllm/pull/234
* @pfldy2850 made their first contribution in https://github.com/vllm-project/vllm/pull/913
* @nelson-liu made their first contribution in https://github.com/vllm-project/vllm/pull/939

**Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.1.4...v0.1.5