Starstruk Logo MarkStarstruk

v0.13.3

xorbitsai/inferencev0.13.3Jul 26, 2024by XprobeBot

AI Summary

Adds GLM4 stream tool calls, support for csg-wukong-chat, mistral-nemo-instruct, mistral-large-instruct, CosyVoice speech, llama-3.1, and rembg for background removal.

Key Highlights

  • GLM4 stream tool call support
  • Llama 3.1 and Mistral model support
  • CosyVoice speech support
  • rembg flexible model for background removal

New Features

  • GLM4 stream tool call
  • csg-wukong-chat support
  • mistral-nemo-instruct support
  • mistral-large-instruct support
  • CosyVoice speech
  • llama-3.1 and llama-3.1-instruct support
  • rembg flexible model

Full Release Notes

# What's new in 0.13.3 (2024-07-26)
These are the changes in inference v0.13.3.
## New features
* FEAT: GLM4 support stream tool call by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1876
* FEAT: support csg-wukong-chat-v0.1 by @qinxuye in https://github.com/xorbitsai/inference/pull/1916
* FEAT: [UI]Add configuration for image and audio models. by @yiboyasss in https://github.com/xorbitsai/inference/pull/1922
* FEAT: support mistral-nemo-instruct by @qinxuye in https://github.com/xorbitsai/inference/pull/1936
* FEAT: CosyVoice speech by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1881
* FEAT: add llama-3.1, llama-3.1-instruct by @Weaxs in https://github.com/xorbitsai/inference/pull/1932
* FEAT: support mistral-large-instruct by @qinxuye in https://github.com/xorbitsai/inference/pull/1944
* Feat: support for llama 3.1 for vllm by @Phoenix500526 in https://github.com/xorbitsai/inference/pull/1935
* FEAT: add rembg flexible model to remove background of image by @qinxuye in https://github.com/xorbitsai/inference/pull/1917
## Enhancements
* ENH: added MLX for llama-3-instruct, codestral, Yi-1.5-chat, internlm2.5-chat by @qinxuye in https://github.com/xorbitsai/inference/pull/1908
* ENH: add gptq for llama-3-instruct by @Phoenix500526 in https://github.com/xorbitsai/inference/pull/1934
## Bug fixes
* BUG: fix inpainting and flexible infer due to inner API change by @qinxuye in https://github.com/xorbitsai/inference/pull/1907
## Documentation
* DOC: update new models to readme by @qinxuye in https://github.com/xorbitsai/inference/pull/1946

## New Contributors
* @Phoenix500526 made their first contribution in https://github.com/xorbitsai/inference/pull/1934

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.13.2...v0.13.3