Starstruk Logo MarkStarstruk

v0.9.3

xorbitsai/inferencev0.9.3Mar 15, 2024by XprobeBot

AI Summary

This release adds the Yi-9B model and image generation functionality. It improves OpenAI API compatibility for the model list and removes quantization limits for Apple Metal.

Key Highlights

  • Add Yi-9B
  • Provide the function of generate image
  • OpenAI compatibility for GET /v1/models
  • Remove quantization limits for Apple Metal

New Features

  • Add Yi-9B
  • Provided the function of generate image
  • Make GET /v1/models compatible with OpenAI API
  • Remove quantization limits for Apple METAL device

Full Release Notes

# What's new in 0.9.3 (2024-03-15)
These are the changes in inference v0.9.3.
## New features
* FEAT: Add Yi-9B by @mujin2 in https://github.com/xorbitsai/inference/pull/1117
* FEAT: Provided the function of generate image. by @hainaweiben in https://github.com/xorbitsai/inference/pull/1047
## Enhancements
* ENH: update cmd help info by @luweizheng in https://github.com/xorbitsai/inference/pull/1106
* ENH: Remove quantization limits for Apple METAL device when running model via `llama-cpp-python` by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1134
* ENH: Make GET /v1/models compatible with OpenAI API. by @notsyncing in https://github.com/xorbitsai/inference/pull/1127
* ENH: support vllm>=0.3.1 by @qinxuye in https://github.com/xorbitsai/inference/pull/1145
## Bug fixes
* BUG: fix the useless fstring. by @mikeshi80 in https://github.com/xorbitsai/inference/pull/1130
* BUG: Fixing the issue of model list loading failure caused by a large number of invalid requests on the model list page. by @wertycn in https://github.com/xorbitsai/inference/pull/1111
* BUG: Fix cache status for embedding, rerank and image models on the web UI by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1135
* BUG: Fix missing information for `xinference registrations` and `xinference list` command by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1140
* BUG: Fix cannot continue to chat after canceling the streaming chat via `ctrl+c` by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1144
## Tests
* TST: Remove testing LLM model creating embedding by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1121
## Documentation
* DOC: fix CPU docker image & refine by @qinxuye in https://github.com/xorbitsai/inference/pull/1118

## New Contributors
* @luweizheng made their first contribution in https://github.com/xorbitsai/inference/pull/1106
* @mujin2 made their first contribution in https://github.com/xorbitsai/inference/pull/1117
* @wertycn made their first contribution in https://github.com/xorbitsai/inference/pull/1111

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.9.2...v0.9.3