Starstruk Logo MarkStarstruk

v1.0.1

xorbitsai/inferencev1.0.1Nov 29, 2024by XprobeBot

AI Summary

This release introduces new model support including Fish speech streaming, Whisper for Mac MLX, and various LLMs like QwQ-32B-Preview and GLM-edge-chat. It also includes bug fixes for embeddings and streaming interfaces.

Key Highlights

  • Added support for QwQ-32B-Preview and GLM-edge-chat models.
  • Introduced Whisper support for Mac MLX.
  • Added sparse vector support for BGE-M3.
  • Implemented guided decoding for vLLM async engine.
  • Fixed GTE-qwen2 Embedding Dimension error and request_limits issues with streaming.

New Features

  • Fish speech stream
  • support sparse vector for bge-m3
  • whisper support for Mac MLX
  • support guided decoding for vllm async engine
  • support QwQ-32B-Preview
  • support glm-edge-chat model

Full Release Notes

# What's new in 1.0.1 (2024-11-29)
These are the changes in inference v1.0.1.
## New features
* FEAT: Fish speech stream by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2562
* FEAT: support sparse vector for bge-m3 by @pengjunfeng11 in https://github.com/xorbitsai/inference/pull/2540
* FEAT: whisper support for Mac MLX by @qinxuye in https://github.com/xorbitsai/inference/pull/2576
* FEAT: support guided decoding for vllm async engine by @wxiwnd in https://github.com/xorbitsai/inference/pull/2391
* FEAT: support QwQ-32B-Preview by @qinxuye in https://github.com/xorbitsai/inference/pull/2602
* FEAT: support glm-edge-chat model by @amumu96 in https://github.com/xorbitsai/inference/pull/2582
## Enhancements
* ENH: Support fish speech reference audio by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2542
## Bug fixes
* BUG: GTE-qwen2 Embedding Dimension error by @cyhasuka in https://github.com/xorbitsai/inference/pull/2565
* BUG: request_limits does not work with streaming interfaces by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/2571
* BUG: Fix Codestral v0.1 URI for Pytorch Format by @danialcheung in https://github.com/xorbitsai/inference/pull/2590
* BUG: Correct the input bytes data by langchain_openai  #2589 by @xiyuan-lee in https://github.com/xorbitsai/inference/pull/2600
## Documentation
* DOC: update builtin models by @qinxuye in https://github.com/xorbitsai/inference/pull/2587

## New Contributors
* @pengjunfeng11 made their first contribution in https://github.com/xorbitsai/inference/pull/2540
* @danialcheung made their first contribution in https://github.com/xorbitsai/inference/pull/2590
* @xiyuan-lee made their first contribution in https://github.com/xorbitsai/inference/pull/2600

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.0.0...v1.0.1