Starstruk Logo MarkStarstruk

v0.9.4

xorbitsai/inferencev0.9.4Mar 21, 2024by XprobeBot

AI Summary

This release adds support for the CodeShell model and introduces SGLang backend support. It also updates vLLM to support the latest models.

Key Highlights

  • Support CodeShell model
  • Supports `sglang` backend
  • vLLM latest models support

New Features

  • Support CodeShell model
  • Supports `sglang` backend
  • vLLM latest models support

Full Release Notes

# What's new in 0.9.4 (2024-03-21)
These are the changes in inference v0.9.4.
## New features
* FEAT: Support CodeShell model by @hainaweiben in https://github.com/xorbitsai/inference/pull/1166
* FEAT: Supports `sglang` backend by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1161
## Enhancements
* ENH: vLLM latest models support by @1572161937 in https://github.com/xorbitsai/inference/pull/1155
## Bug fixes
* BUG: remove `best_of` from benchmark by @qinxuye in https://github.com/xorbitsai/inference/pull/1150
* BUG: fix  _eval_qwen_chat_arguments parsing problem by @channingxiao18 in https://github.com/xorbitsai/inference/pull/1098
* BUG: Fix OpenAI compatibility issue during chat by @mujin2 in https://github.com/xorbitsai/inference/pull/1159
## Documentation
* DOC: Update doc by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1156
## Others
* Chore: add assign workflow by @qinxuye in https://github.com/xorbitsai/inference/pull/1131

## New Contributors
* @channingxiao18 made their first contribution in https://github.com/xorbitsai/inference/pull/1098
* @1572161937 made their first contribution in https://github.com/xorbitsai/inference/pull/1155

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.9.3...v0.9.4