Starstruk Logo MarkStarstruk

v1.10.1

xorbitsai/inferencev1.10.1Oct 1, 2025by XprobeBot

AI Summary

This release focuses on OpenAI API enhancements, UI improvements, and support for several new models including IndexTTS2, Qwen3-VL, and Qwen3-Next.

Key Highlights

  • Added OpenAI API support for sglang JSON structured output
  • Enhanced UI with request_limits parameter support
  • Added support for listing flexible models via webui and cmdline

New Features

  • OpenAI API support for sglang JSON structured output
  • UI support for request_limits parameter
  • Flexible model listing via webui and cmdline
  • IndexTTS2 model support
  • Qwen3-VL model support
  • Qwen3-Next model support

Full Release Notes

# What's new in 1.10.1 (2025-10-01)
These are the changes in inference v1.10.1.
## New features
* FEAT: Openai API support sglang json structured output by @OliverBryant in https://github.com/xorbitsai/inference/pull/4070
* FEAT: [UI] support request_limits parameter for all models. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4081
* FEAT: support list flexible model via webui and cmdline by @leslie2046 in https://github.com/xorbitsai/inference/pull/4085
* FEAT: [model] Support IndexTTS2 by @OliverBryant in https://github.com/xorbitsai/inference/pull/4078
* FEAT: [UI] support dynamic download_hub detection. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4082
* FEAT: [model] qwen-image-edit-2509 by @qinxuye in https://github.com/xorbitsai/inference/pull/4099
* FEAT: [model] baichuan-M2 by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4107
* FEAT: [model] Support Qwen3-VL by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4112
* FEAT: [model] Support Qwen3-Next by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4113
## Enhancements
* ENH: optimize MPS on Mac for Qwen2.5-VL by @SolardiaX in https://github.com/xorbitsai/inference/pull/3524
* ENH: deepseek-r1-0528 support tool_calls by @amumu96 in https://github.com/xorbitsai/inference/pull/4106
* BLD: update funasr by @leslie2046 in https://github.com/xorbitsai/inference/pull/4062
* BLD: Update Dockerfile.cu128 by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4114
* REF: [UI] refactor the launch model page. by @yiboyasss in https://github.com/xorbitsai/inference/pull/3940
## Bug fixes
* BUG: Optimize rerank model lookup logic and add support for video model type by @amumu96 in https://github.com/xorbitsai/inference/pull/4063
* BUG: Fix seed-oss required VLLM_VERSION by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4071
* BUG: fix register_model when model name is duplicated by @llyycchhee in https://github.com/xorbitsai/inference/pull/4076
* BUG: [UI] fix the custom model drawer component could not be opened. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4089
* BUG: Fix the issue where registered models cannot use tools by @amumu96 in https://github.com/xorbitsai/inference/pull/4100
* BUG: fix finish_reason field handling logic by @amumu96 in https://github.com/xorbitsai/inference/pull/4105
* BUG: vllm structured output compatibility by @OliverBryant in https://github.com/xorbitsai/inference/pull/4111
## Documentation
* DOC: Update README.md about MaxKB by @chixq in https://github.com/xorbitsai/inference/pull/3771

## New Contributors
* @chixq made their first contribution in https://github.com/xorbitsai/inference/pull/3771
* @SolardiaX made their first contribution in https://github.com/xorbitsai/inference/pull/3524

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.10.0...v1.10.1