Starstruk Logo MarkStarstruk

v1.1.0

xorbitsai/inferencev1.1.0Dec 13, 2024by XprobeBot

AI Summary

This release introduces F5 TTS support, VL model support for MLX, CLIP model support, and Llama-3.3-Instruct support. It also enhances error messages and unifies prompt text for speech models.

Key Highlights

  • Support for F5 TTS.
  • Support for VL models on MLX.
  • Support for CLIP model.
  • Support for Llama-3.3-Instruct.
  • Enhanced error messages for incorrect parameters.

New Features

  • Support F5 TTS
  • [UI] Add a hint for model running
  • support VL models for MLX
  • Add support for CLIP model
  • support llama-3.3-instruct

Full Release Notes

# What's new in 1.1.0 (2024-12-13)
These are the changes in inference v1.1.0.
## New features
* FEAT: Support F5 TTS by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2626
* FEAT: [UI] Add a hint for model running. by @yiboyasss in https://github.com/xorbitsai/inference/pull/2657
* FEAT: support VL models for MLX by @qinxuye in https://github.com/xorbitsai/inference/pull/2638
* FEAT: Add support for CLIP model by @Second222None in https://github.com/xorbitsai/inference/pull/2637
* FEAT: support llama-3.3-instruct by @qinxuye in https://github.com/xorbitsai/inference/pull/2661
## Enhancements
* ENH: Optimize error message when user parameters are passed incorrectly by @namecd in https://github.com/xorbitsai/inference/pull/2623
* ENH: bypass the sampling parameter skip_special_tokens to vLLM backend by @zjuyzj in https://github.com/xorbitsai/inference/pull/2655
* ENH: unify prompt_text as cosyvoice for fish speech by @qinxuye in https://github.com/xorbitsai/inference/pull/2658
* ENH: Update glm4 chat model to new weights by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2660
* ENH: upgrade sglang in Docker by @amumu96 in https://github.com/xorbitsai/inference/pull/2668
## Bug fixes
* BUG: Cleanup Isolation tasks by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2603
* BUG: fix qwq gguf download hub for modelscope by @redreamality in https://github.com/xorbitsai/inference/pull/2647
* BUG: fix ImportError when optional dependency FlagEmbedding is not installed by @zjuyzj in https://github.com/xorbitsai/inference/pull/2649
* BUG: use stream_generate in MLX by @qinxuye in https://github.com/xorbitsai/inference/pull/2635
* BUG: `stop` parameter leads to failure with `transformers` backend by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/2663
* BUG: fix FishSpeech Negative code found by @themanforfree in https://github.com/xorbitsai/inference/pull/2667
## Documentation
* DOC: update new models by @qinxuye in https://github.com/xorbitsai/inference/pull/2632
* DOC: add doc about offline usage for SenseVoiceSmall by @qinxuye in https://github.com/xorbitsai/inference/pull/2654
## Others
* FIX: fix launching bge-m3 with hybrid mode by @pengjunfeng11 in https://github.com/xorbitsai/inference/pull/2641

## New Contributors
* @namecd made their first contribution in https://github.com/xorbitsai/inference/pull/2623
* @redreamality made their first contribution in https://github.com/xorbitsai/inference/pull/2647
* @Second222None made their first contribution in https://github.com/xorbitsai/inference/pull/2637
* @themanforfree made their first contribution in https://github.com/xorbitsai/inference/pull/2667

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.0.1...v1.1.0