Starstruk Logo MarkStarstruk

v2.5.0

Helicone/heliconev2.5.0Apr 13, 2026by XprobeBot

AI Summary

A release with audio generation and model support for xorbitsai/inference.

Key Highlights

  • sglang support for qwen3.5
  • Reconnect and reconstruct model replicas after supervisor restart
  • Audio support for qwen3-tts with multiple variants
  • LocalStorage management for model deploy configuration
  • gemma-4 support
  • Auto-detect PyTorch CUDA version for venv setup

New Features

  • sglang support for qwen3.5
  • Model replica reconstruction after supervisor restart
  • Audio support for qwen3-tts (12Hz variants)
  • LocalStorage management for model config
  • gemma-4 support
  • Auto-detect PyTorch CUDA version

Full Release Notes

# What's new in 2.5.0 (2026-04-13)
These are the changes in inference v2.5.0.
## New features
* feat(sglang): support qwen3.5 by @llyycchhee in https://github.com/xorbitsai/inference/pull/4763
* FEAT: reconnect and reconstrcut model replicas after restart supervisor by @leslie2046 in https://github.com/xorbitsai/inference/pull/4731
* feat(audio): support qwen3-tts by @llyycchhee in https://github.com/xorbitsai/inference/pull/4781
* FEAT: [model] Qwen3-TTS-12Hz-1.7B-Base support by @llyycchhee in https://github.com/xorbitsai/inference/pull/4776
* FEAT: [model] Qwen3-TTS-12Hz-0.6B-Base support by @llyycchhee in https://github.com/xorbitsai/inference/pull/4777
* FEAT: [model] Qwen3-TTS-12Hz-1.7B-CustomVoice support by @llyycchhee in https://github.com/xorbitsai/inference/pull/4778
* FEAT: [model] Qwen3-TTS-12Hz-0.6B-CustomVoice support by @llyycchhee in https://github.com/xorbitsai/inference/pull/4779
* FEAT: [model] Qwen3-TTS-12Hz-1.7B-VoiceDesign support by @llyycchhee in https://github.com/xorbitsai/inference/pull/4780
* FEAT(webui): add localstorage management for model deploy configuration by @leslie2046 in https://github.com/xorbitsai/inference/pull/4739
* FEAT: [model] gemma-4 support by @qinxuye in https://github.com/xorbitsai/inference/pull/4768
## Enhancements
* ENH: update model "DeepSeek-OCR" JSON by @amumu96 in https://github.com/xorbitsai/inference/pull/4751
* ENH: update 2 models JSON ("Ernie4.5", "qwen3.5") by @XprobeBot in https://github.com/xorbitsai/inference/pull/4754
* ENH: update model "DeepSeek-V3.2" JSON by @amumu96 in https://github.com/xorbitsai/inference/pull/4762
* ENH: update 2 models JSON ("Qwen3-ASR-0.6B", "Qwen3-ASR-1.7B") by @qinxuye in https://github.com/xorbitsai/inference/pull/4765
* ENH: auto-detect PyTorch CUDA version for virtual environment setup by @qinxuye in https://github.com/xorbitsai/inference/pull/4766
* ENH: update model "jina-embeddings-v4" JSON by @qinxuye in https://github.com/xorbitsai/inference/pull/4775
* ENH: Optimize worker details for deployment progress tooltip. by @leslie2046 in https://github.com/xorbitsai/inference/pull/4746
* ENH: update model "qwen3.5" JSON by @llyycchhee in https://github.com/xorbitsai/inference/pull/4782
* ENH: update 2 models JSON ("Kokoro-82M-v1.1-zh", "Kokoro-82M") by @qinxuye in https://github.com/xorbitsai/inference/pull/4795
* ENH: update model "gemma-3-it" JSON by @qinxuye in https://github.com/xorbitsai/inference/pull/4794
* ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4796
* ENH: add lightweight heartbeat mechanism for worker liveness detection by @qinxuye in https://github.com/xorbitsai/inference/pull/4785
* ENH: update model "ChatTTS" JSON by @qinxuye in https://github.com/xorbitsai/inference/pull/4793
* bld: Fix the front-end UI access issue for aarch64 image by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4743
* bld: Fix the front-end UI access issue for aarch64 image by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4749
* bld: Fix the front-end UI access issue by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4758
* BLD: limit gptqmodel installation to specified version by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4798
## Bug fixes
* fix: use constant-time comparison for auth credentials (CWE-208) by @spidershield-contrib in https://github.com/xorbitsai/inference/pull/4734
* bug: fix qwen3 reranker vllm precision by @ZhikaiGuo960110 in https://github.com/xorbitsai/inference/pull/4747
* fix: add variable to control template for Qwen3 Reranker Family by @ZhikaiGuo960110 in https://github.com/xorbitsai/inference/pull/4752
* BUG: Fix Qwen3.5 wrong <think> tag in streaming API by @la1ty in https://github.com/xorbitsai/inference/pull/4759
* BUG: Fix Jinja template error for models using {% break %} tag (e.g. ……Kimi K2.5) by @amumu96 in https://github.com/xorbitsai/inference/pull/4770
* BUG: fix qwen3-vl embedding model for vllm engine by @llyycchhee in https://github.com/xorbitsai/inference/pull/4783
## Documentation
* DOC: add v2.4.0 release notes by @qinxuye in https://github.com/xorbitsai/inference/pull/4740
## Others
* Fix #4597: [Bug] v2.0.0 Docker image: ImportError (circular import) a... by @JiwaniZakir in https://github.com/xorbitsai/inference/pull/4757

## New Contributors
* @spidershield-contrib made their first contribution in https://github.com/xorbitsai/inference/pull/4734
* @JiwaniZakir made their first contribution in https://github.com/xorbitsai/inference/pull/4757

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v2.4.0...v2.5.0