Starstruk Logo MarkStarstruk

v0.10.0

xorbitsai/inferencev0.10.0Mar 29, 2024by XprobeBot

AI Summary

A significant update introducing the Web UI for audio models, OmniLMM support, and vLLM support for DeepSeek models. It also introduces an OAuth system.

Key Highlights

  • Launch UI of audio model
  • Support for OmniLMM chat model
  • vLLM support for DeepSeek models
  • OAuth system supporting API-key
  • Force to specify worker IP and GPU idx

New Features

  • launch UI of audio model
  • Supports `OmniLMM` chat model
  • Added vllm support for deepseek models
  • force to specify worker ip and gpu idx when launching models
  • OAuth system supports api-key
  • Support deepseek vl

Full Release Notes

# What's new in 0.10.0 (2024-03-29)
These are the changes in inference v0.10.0.
## New features
* FEAT: launch UI of audio model. by @hainaweiben in https://github.com/xorbitsai/inference/pull/1102
* FEAT: Supports `OmniLMM` chat model by @hainaweiben in https://github.com/xorbitsai/inference/pull/1171
* FEAT: Added vllm support for deepseek models by @ivanzfb in https://github.com/xorbitsai/inference/pull/1200
* FEAT: force to specify worker ip and gpu idx when launching models by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1195
* FEAT: OAuth system supports api-key by @Ago327 in https://github.com/xorbitsai/inference/pull/1168
* FEAT: Support deepseek vl by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1175
* FEAT: support some builtin new models by @mujin2 in https://github.com/xorbitsai/inference/pull/1204
## Enhancements
* BLD: add autoawq in setup by @utopia2077 in https://github.com/xorbitsai/inference/pull/1190
## Bug fixes
* BUG: Fix the incorrect model interface address caused a 307 redirect to HTTP, blocking the request and preventing the display of the model list. by @wertycn in https://github.com/xorbitsai/inference/pull/1182
* BUG: fix doc fail introduced by #1171 & update readme by @qinxuye in https://github.com/xorbitsai/inference/pull/1203
* BUG: Increase validator types for thie 'input' parameter of embeddings to match OpenAI API by @Minamiyama in https://github.com/xorbitsai/inference/pull/1201
## Documentation
* DOC: internal design by @1572161937 in https://github.com/xorbitsai/inference/pull/1178
* Doc: update readme and models doc by @qinxuye in https://github.com/xorbitsai/inference/pull/1176
* DOC: Doc for oauth system with api-key by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1210

## New Contributors
* @utopia2077 made their first contribution in https://github.com/xorbitsai/inference/pull/1190
* @ivanzfb made their first contribution in https://github.com/xorbitsai/inference/pull/1200

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.9.4...v0.10.0