Starstruk Logo MarkStarstruk

v1.16.0

xorbitsai/inferencev1.16.0Dec 27, 2025by XprobeBot

AI Summary

Version 1.16.0 adds DeepSeek-V3.2-Exp support, Fun-ASR speech recognition models, Qwen-Image-Layered, and MiniMaxM2. Introduces vacc support, continuous batching for MLX chat models, and rerank async batch functionality.

Key Highlights

  • DeepSeek-V3.2-Exp model support
  • Fun-ASR-Nano-2512 and Fun-ASR-MLT-Nano-2512 speech recognition
  • Qwen-Image-Layered model
  • vacc support with VLM support
  • Continuous batching for MLX chat models
  • Rerank async batch support
  • Architectures field for LLM model launch
  • Removed Python 3.9 support

Breaking Changes

  • Python 3.9 support removed

New Features

  • DeepSeek-V3.2-Exp
  • Fun-ASR-Nano-2512
  • Fun-ASR-MLT-Nano-2512
  • Qwen-Image-Layered
  • MiniMaxM2 (VLLM backend)
  • vacc support
  • continuous batching for MLX
  • rerank async batch

Full Release Notes

# What's new in 1.16.0 (2025-12-27)
These are the changes in inference v1.16.0.
## New features
* FEAT: [model] DeepSeek-V3.2-Exp support by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4374
* FEAT:Add vLLM backend support for DeepSeek-V3.2 by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4377
* FEAT:Add vLLM backend support for DeepSeek-V3.2-Exp. by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4375
* FEAT: vacc support by @ZhikaiGuo960110 in https://github.com/xorbitsai/inference/pull/4382
* FEAT: support vlm for vacc by @ZhikaiGuo960110 in https://github.com/xorbitsai/inference/pull/4385
* FEAT: [model] Fun-ASR-Nano-2512 support by @leslie2046 in https://github.com/xorbitsai/inference/pull/4397
* FEAT: [model] Qwen-Image-Layered support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4395
* FEAT: [model] Fun-ASR-MLT-Nano-2512 support by @leslie2046 in https://github.com/xorbitsai/inference/pull/4398
* FEAT: continuous batching support for MLX chat models by @qinxuye in https://github.com/xorbitsai/inference/pull/4403
* FEAT: Add the architectures field for llm model launch by @OliverBryant in https://github.com/xorbitsai/inference/pull/4405
* FEAT: [UI] image models support configuration via environment variables and custom parameters. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4413
* FEAT: support rerank async batch by @llyycchhee in https://github.com/xorbitsai/inference/pull/4414
* FEAT:Support VLLM backend for MiniMaxM2ForCausalLM by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4412
## Enhancements
* ENH: fix assigning replica to make gpu idxes assigned continuous by @ZhikaiGuo960110 in https://github.com/xorbitsai/inference/pull/4370
* ENH: update model "DeepSeek-V3.2" JSON by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4381
* ENH: update model "glm-4.5" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4383
* ENH: update 2 models JSON ("glm-4.1v-thinking", "glm-4.5v") by @OliverBryant in https://github.com/xorbitsai/inference/pull/4384
* ENH: support torchaudio 2.9.0 by @llyycchhee in https://github.com/xorbitsai/inference/pull/4390
* ENH: update 3 models JSON ("llama-2-chat", "llama-3", "llama-3-instruct") by @OliverBryant in https://github.com/xorbitsai/inference/pull/4400
* ENH: update 4 models JSON ("llama-3.1", "llama-3.1-instruct", "llama-3.2-vision-instruct", ... +1 more) by @OliverBryant in https://github.com/xorbitsai/inference/pull/4401
* ENH: update model "jina-embeddings-v3" JSON by @XprobeBot in https://github.com/xorbitsai/inference/pull/4404
* ENH: update models JSON [audio, embedding, image, llm, video] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4407
* ENH: update models JSON [audio, image] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4408
* ENH: update model "Z-Image-Turbo" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4409
* ENH: update 2 models JSON ("DeepSeek-V3.2", "DeepSeek-V3.2-Exp") by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4392
* ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4415
* BLD: remove python 3.9 support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4387
* BLD: Update Dockerfile to 12.9 to use VLLM v0.11.2 version by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4393
## Bug fixes
* BUG: fix PaddleOCR-VL output by @leslie2046 in https://github.com/xorbitsai/inference/pull/4368
* BUG: custom embedding and rerank model analysis error by @OliverBryant in https://github.com/xorbitsai/inference/pull/4367
* BUG: cannot launch model on cpu && multi workers launch error by @OliverBryant in https://github.com/xorbitsai/inference/pull/4361
* BUG: OCR API return is null && add doc for how to modify model_size by @OliverBryant in https://github.com/xorbitsai/inference/pull/4331
* BUG: fix n_gpu parameter by @OliverBryant in https://github.com/xorbitsai/inference/pull/4411
## Documentation
* DOC: update new models and release notes for v1.15.0 by @qinxuye in https://github.com/xorbitsai/inference/pull/4359


**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.15.0...v1.16.0