Starstruk Logo MarkStarstruk

v0.9.0

xorbitsai/inferencev0.9.0Feb 22, 2024by XprobeBot

AI Summary

The v0.9.0 major release introduces Intel GPU support and refactors device-related code. It also adds the Gemma model series and improves the UI to show cluster resources.

Key Highlights

  • Refactored device code and added initial Intel GPU support
  • Added support for the Gemma series model family
  • Updated the web UI to display cluster resource information and replica counts
  • Fixed chat completion errors related to message indexing

New Features

  • Refactor device related code and add initial Intel GPU support
  • Support gemma series model

Full Release Notes

# What's new in 0.9.0 (2024-02-22)
These are the changes in inference v0.9.0.
## New features
* FEAT: Refactor device related code and add initial Intel GPU support by @notsyncing in https://github.com/xorbitsai/inference/pull/968
* FEAT: Support gemma series model by @aresnow1 in https://github.com/xorbitsai/inference/pull/1024
## Enhancements
* ENH: [UI] Supports `replica` when launching LLM models by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1011
* ENH: [UI] Show cluster resource information by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1015
## Bug fixes
* BUG: fix chat completion error when indexing body.messages by @fffonion in https://github.com/xorbitsai/inference/pull/1008
* BUG: Fix cache sd 1.5 error by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1013
* BUG: fix typo in modelscope llama-2-13b-chat-GGUF by @qinxuye in https://github.com/xorbitsai/inference/pull/1026
* BUG: Fix missing qwen 1.5 7b gguf by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1027
## Documentation
* DOC: Polish model operation command doc by @onesuper in https://github.com/xorbitsai/inference/pull/1000
* DOC: Fix note on secret_key generation and algorithm selection for OAuth2 by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1012

## New Contributors
* @fffonion made their first contribution in https://github.com/xorbitsai/inference/pull/1008
* @notsyncing made their first contribution in https://github.com/xorbitsai/inference/pull/968

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.8.5...v0.9.0