Starstruk Logo MarkStarstruk

v0.9.1

xorbitsai/inferencev0.9.1Mar 1, 2024by XprobeBot

AI Summary

v0.9.1 adds CPU-only Docker support and improves model downloading from ModelScope. It also refines the RESTful client and command-line interface for better usability.

Key Highlights

  • Released a Docker image for CPU-only environments
  • Enabled downloading Gemma models directly from ModelScope
  • Enhanced RESTful client to support multiple system prompts for chat
  • Added extra parameters to the `xinference launch` command line

New Features

  • Docker for CPU only
  • Support downloading gemma from modelscope

Full Release Notes

# What's new in 0.9.1 (2024-03-01)
These are the changes in inference v0.9.1.
## New features
* FEAT: Docker for cpu only by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1068
## Enhancements
* ENH: Support downloading gemma from modelscope by @aresnow1 in https://github.com/xorbitsai/inference/pull/1035
* ENH: [UI] Setting `quantization` when registering LLM by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1040
* ENH: Restful client supports multiple system prompts for chat by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1056
* ENH: supports disabling worker reporting status by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1057
* ENH: Extra params for `xinference launch` command line by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1048
## Bug fixes
* BUG: Fix some models that cannot download from `modelscope` by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1066
* BUG: Fix early truncation due to `max_token` being default to `16` instead of `1024` by @ZhangTianrong in https://github.com/xorbitsai/inference/pull/1061
## Documentation
* DOC: Update readme by @qinxuye in https://github.com/xorbitsai/inference/pull/1045
* DOC: Fix readme by @qinxuye in https://github.com/xorbitsai/inference/pull/1054
* DOC: Fix wechat links by @qinxuye in https://github.com/xorbitsai/inference/pull/1055

## New Contributors
* @ZhangTianrong made their first contribution in https://github.com/xorbitsai/inference/pull/1061

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.9.0...v0.9.1