Starstruk Logo MarkStarstruk

v2.3.0

xorbitsai/inferencev2.3.0Mar 13, 2026by XprobeBot

AI Summary

This release adds Qwen3.5 support for vLLM and transformers, introduces seed and repetition_penalty parameters, removes qwen2-audio model, and fixes various bugs related to worker initialization and vLLM.

Key Highlights

  • Support qwen-3.5 for vllm and transformers
  • Add seed and repetition_penalty parameter for precision test
  • Remove qwen2-audio model
  • Fix worker initialization error
  • Fix vLLM embedding check for qwen3-vl-embedding

New Features

  • Support qwen-3.5 for vllm
  • Add seed and repetition_penalty parameter for precision test
  • Remove qwen2-audio model

Full Release Notes

# What's new in 2.3.0 (2026-03-13)
These are the changes in inference v2.3.0.
## New features
* FEAT: support qwen-3.5 for vllm by @llyycchhee in https://github.com/xorbitsai/inference/pull/4656
* FEAT: add seed and repeptition_penalty parameter for precision test by @ZhikaiGuo960110 in https://github.com/xorbitsai/inference/pull/4684
* FEAT: [model] qwen2-audio removed by @ZhikaiGuo960110 in https://github.com/xorbitsai/inference/pull/4683
## Enhancements
* ENH: update 2 models JSON ("qwen3.5", "glm-5") by @llyycchhee in https://github.com/xorbitsai/inference/pull/4655
* ENH: update model "MiniMax-M2.5" JSON by @llyycchhee in https://github.com/xorbitsai/inference/pull/4663
* ENH: update model "qwen3.5" JSON by @llyycchhee in https://github.com/xorbitsai/inference/pull/4661
* ENH: update model "qwen3.5" JSON by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4672
* ENH: update 2 models JSON ("glm-5", "Kimi-K2.5") by @llyycchhee in https://github.com/xorbitsai/inference/pull/4662
* ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4682
* ENH: support transformers for qwen 3.5 by @llyycchhee in https://github.com/xorbitsai/inference/pull/4685
* ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4686
* BLD: [CI] fix windows runner SSL can't found by @llyycchhee in https://github.com/xorbitsai/inference/pull/4627
* REF: Implement REST API dependency injection and response handling by @amumu96 in https://github.com/xorbitsai/inference/pull/4620
* REF: extract require_model utility to reduce code duplication by @amumu96 in https://github.com/xorbitsai/inference/pull/4677
## Bug fixes
* BUG: fix error `WorkerWrapperBase.__init__() got multiple values for argument 'rpc_rank'` by @llyycchhee in https://github.com/xorbitsai/inference/pull/4649
* BUG: fix vLLM embedding check for qwen3-vl-embedding by @ace-xc in https://github.com/xorbitsai/inference/pull/4647
* FIX: update the QR code URL by @yiboyasss in https://github.com/xorbitsai/inference/pull/4668
* BUG: fix chat for multiple gpus by @llyycchhee in https://github.com/xorbitsai/inference/pull/4671
* BUG: [UI] initialize formData with default values from modelFormConfig. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4678
* BUG: fix qwen 3.5 vllm since no generation_config.json exists by @llyycchhee in https://github.com/xorbitsai/inference/pull/4681
## Documentation
* DOC: add v2.2.0 release notes by @qinxuye in https://github.com/xorbitsai/inference/pull/4643
* DOC: add missing periods in docstrings by @Jah-yee in https://github.com/xorbitsai/inference/pull/4669

## New Contributors
* @Jah-yee made their first contribution in https://github.com/xorbitsai/inference/pull/4669

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v2.2.0...v2.3.0