Starstruk Logo MarkStarstruk

v0.12.1

xorbitsai/inferencev0.12.1Jun 14, 2024by XprobeBot

AI Summary

Adds tool call support for Qwen2 and GLM4-chat, along with a new method to download models from CSGHub. The UI also gains viewing and deleting cache data capabilities, and GLM-4V gains quantization support.

Key Highlights

  • Tool call support for Qwen2 and GLM4-chat
  • CSGHub model downloading support
  • Cache viewing and deletion UI
  • Quantization support for GLM-4V

New Features

  • qwen2-instruct support tool call
  • Added a method to download models from csghub
  • glm4-chat support tool call
  • Supports viewing and deleting cache data
  • quantization for glm-4v

Full Release Notes

# What's new in 0.12.1 (2024-06-14)
These are the changes in inference v0.12.1.
## New features
* FEAT: qwen2-instruct support tool call by @ayhhyhh in https://github.com/xorbitsai/inference/pull/1631
* FEAT: Added a method to download models from csghub.  by @hainaweiben in https://github.com/xorbitsai/inference/pull/1627
* FEAT: glm4-chat support tool call by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1617
* FEAT: [UI] Supports viewing and deleting cache data. by @yiboyasss in https://github.com/xorbitsai/inference/pull/1637
## Enhancements
* ENH: modelscope for audio models by @Minamiyama in https://github.com/xorbitsai/inference/pull/1607
* ENH: Supports `generate` interface for continuous batching by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1621
* ENH: quantization for glm-4v by @Minamiyama in https://github.com/xorbitsai/inference/pull/1610
## Bug fixes
* BUG: Fix wheel package missing thirdparty ChatTTS by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1606
* BUG: fix XINFERENCE_MODEL_SRC behavior by @LukeWang-Plus in https://github.com/xorbitsai/inference/pull/1616
* BUG: Filtering Step for Streaming Responses to Qwen's Tool Calls when using vLLM by @zhanghx0905 in https://github.com/xorbitsai/inference/pull/1598
## Others
* Remove selected cache models by @hainaweiben in https://github.com/xorbitsai/inference/pull/1613

## New Contributors
* @LukeWang-Plus made their first contribution in https://github.com/xorbitsai/inference/pull/1616
* @ayhhyhh made their first contribution in https://github.com/xorbitsai/inference/pull/1631

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.12.0...v0.12.1