Starstruk Logo MarkStarstruk

v0.14.1

xorbitsai/inferencev0.14.1Aug 9, 2024by XprobeBot

AI Summary

Adds support for SenseVoice audio-to-text, Flux.1 models, Kolors image model, CogVideoX video model, and MiniCPM-v-2_6. Enhances internal server error handling and adds streaming option to Benchmark.

Key Highlights

  • SenseVoice audio-to-text model support
  • Flux.1-schnell & Flux.1-dev support
  • CogVideoX video model support
  • MiniCPM-v-2_6 support

New Features

  • SenseVoice audio-to-text support
  • Flux.1-schnell & Flux.1-dev support
  • Kolors image model support
  • CogVideoX video model support
  • MiniCPM-v-2_6 support

Full Release Notes

# What's new in 0.14.1 (2024-08-09)
These are the changes in inference v0.14.1.
## New features
* FEAT: support SenseVoice audio-to-text model by @qinxuye in https://github.com/xorbitsai/inference/pull/2008
* FEAT: support flux.1-schnell & flux.1-dev by @qinxuye in https://github.com/xorbitsai/inference/pull/2007
* FEAT: support kolors image model by @qinxuye in https://github.com/xorbitsai/inference/pull/2028
* FEAT: Add support for llama-3.1-instruct 405B model by @frostyplanet in https://github.com/xorbitsai/inference/pull/2025
* FEAT: Support CogVideoX video model by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2049
* FEAT: Support MiniCPM-v-2_6 by @Minamiyama in https://github.com/xorbitsai/inference/pull/2031
## Enhancements
* ENH: Improve internal server error by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2009
* ENH: Add `stream` option in Benchmark by @Dawnfz-Lenfeng in https://github.com/xorbitsai/inference/pull/2038
* ENH: optimize availability of vLLM by @qinxuye in https://github.com/xorbitsai/inference/pull/2046
* ENH: [worker] Allow init supervisor_ref lazy by @frostyplanet in https://github.com/xorbitsai/inference/pull/1958
* ENH: optimize performance of sglang by @qinxuye in https://github.com/xorbitsai/inference/pull/2050
* REF: Mark `Deprecate` for `prompt`, `system_prompt` and `chat_history` parameters in `chat` client interface by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/2043
## Bug fixes
* BUG: fix flexible model register in worker by @frostyplanet in https://github.com/xorbitsai/inference/pull/2011
* BUG: [UI] Fix the 'model_path' bug. by @yiboyasss in https://github.com/xorbitsai/inference/pull/2015
* BUG: fix custom embedding launch error by @amumu96 in https://github.com/xorbitsai/inference/pull/2016
## Tests
* TST: Fix some dependency version issues by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/2042
## Documentation
* DOC: Directly launch custom model by `model_path` by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/2047
* DOC: fix typo in README by @ArtificialZeng in https://github.com/xorbitsai/inference/pull/2048
## Others
* CHORE: Increased frequency of issue processing by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/2024

## New Contributors
* @ArtificialZeng made their first contribution in https://github.com/xorbitsai/inference/pull/2048
* @Dawnfz-Lenfeng made their first contribution in https://github.com/xorbitsai/inference/pull/2038

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.14.0...v0.14.1