Starstruk Logo MarkStarstruk

v3.1.0

xorbitsai/inferencev3.1.0Jul 31, 2026by XprobeBot

AI Summary

Version 3.1.0 introduces significant enhancements to the user interface, including SSO/OIDC login support and multi-dashboard monitoring capabilities. It expands model capabilities by adding support for MiniMax-M3 and new speculative decoding features, alongside new image and audio engines. The release also focuses on stability by addressing numerous bug fixes related to worker management, dependency conflicts, and security vulnerabilities.

Key Highlights

  • Added SSO/OIDC login button for secure authentication
  • Implemented multi-dashboard monitor center and multi-node selection capabilities
  • Expanded model support with MiniMax-M3 and new speculative decoding features
  • Added new image and audio engines using sglang and vllm
  • Fixed critical stability issues including Slow HTTP DoS mitigation and worker stale state handling

New Features

  • SSO/OIDC login button to login page
  • Launch model worker multi select
  • Batch Qwen3-ASR transcriptions
  • Auto-refresh to audit center
  • Multi-dashboard monitor center with admin config dialog
  • Auto-reload shell routes on export directory change
  • Populate logprobs/top_logprobs in completion responses
  • Speculative decoding with paired drafter checkpoints
  • Image and audio sglang/vllm engines
  • Support for MiniMax-M3 model

Full Release Notes

# What's new in 3.1.0 (2026-07-31)
These are the changes in inference v3.1.0.
## New features
* feat(ui): add SSO/OIDC login button to login page by @m199369309 in https://github.com/xorbitsai/inference/pull/5218
* feat: launch model worker multi select by @maoyuehui in https://github.com/xorbitsai/inference/pull/5222
* feat(audio): batch Qwen3-ASR transcriptions by @qinxuye in https://github.com/xorbitsai/inference/pull/5172
* feat(webui): add auto-refresh to audit center by @m199369309 in https://github.com/xorbitsai/inference/pull/5226
* feat(webui): show API key name and model ID columns in audit center by @m199369309 in https://github.com/xorbitsai/inference/pull/5229
* feat(webui): multi-dashboard monitor center with admin config dialog by @m199369309 in https://github.com/xorbitsai/inference/pull/5225
* feat(frontend): auto-reload shell routes on export directory change by @m199369309 in https://github.com/xorbitsai/inference/pull/5243
* feat(vllm): populate logprobs/top_logprobs in completion responses by @SuperMarioYL in https://github.com/xorbitsai/inference/pull/5245
* feat(webui): support multi-node selection in log center by @m199369309 in https://github.com/xorbitsai/inference/pull/5246
* feat: speculative decoding with paired drafter checkpoints by @qinxuye in https://github.com/xorbitsai/inference/pull/5241
* feat(engine): [image, audio] sglang image vllm asr engines by @OliverBryant in https://github.com/xorbitsai/inference/pull/5121
* feat(webui): add missing permission scopes (models:register, logs:list, monitor:view) to UI by @m199369309 in https://github.com/xorbitsai/inference/pull/5254
* FEAT: [model] MiniMax-M3 support by @llyycchhee in https://github.com/xorbitsai/inference/pull/5258
## Enhancements
* ENH: update models JSON [embedding, rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5214
* ENH: update models JSON [audio] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5233
* ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5236
* ENH: display replica launch status in responsive grid by @leslie2046 in https://github.com/xorbitsai/inference/pull/5239
* ENH: update models JSON [audio, embedding, image, llm, rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5251
* ENH: adopt xoscar 0.9.7 graceful #system_*# fallback by @OliverBryant in https://github.com/xorbitsai/inference/pull/5228
* ENH: update model "MiniMax-M3" JSON by @llyycchhee in https://github.com/xorbitsai/inference/pull/5263
* BLD: build the aarch64 image from the unified slim Dockerfile by @OliverBryant in https://github.com/xorbitsai/inference/pull/5209
* BLD: improve Docker build caching by @leslie2046 in https://github.com/xorbitsai/inference/pull/5231
* BLD: require peft>=0.18.0 for transformers 5.x compatibility by @OliverBryant in https://github.com/xorbitsai/inference/pull/5260
## Bug fixes
* fix(embedding/rerank): fix sentence_transformers load failures from torch-ecosystem package mismatches by @OliverBryant in https://github.com/xorbitsai/inference/pull/5210
* fix(webui): show model engine on running-model detail page by @OliverBryant in https://github.com/xorbitsai/inference/pull/5211
* fix(pypiserver): emit model source paths with forward slashes by @OliverBryant in https://github.com/xorbitsai/inference/pull/5216
* fix: (UI) Update the label of n_gpu except for LLM and Image types by @maoyuehui in https://github.com/xorbitsai/inference/pull/5217
* FIX: mitigate Slow HTTP DoS (Slowloris) by enforcing a request receipt deadline by @OliverBryant in https://github.com/xorbitsai/inference/pull/5213
* fix(api): cast replica to int to prevent TypeError on multi-replica launch by @m199369309 in https://github.com/xorbitsai/inference/pull/5220
* fix(venv): pin torch to system version to prevent torchvision ABI crash on relaunch by @OliverBryant in https://github.com/xorbitsai/inference/pull/5212
* BUG: worker keeps stale model state after a launch fails during async engine load by @OliverBryant in https://github.com/xorbitsai/inference/pull/5193
* fix(audio/indextts2): declare the missing IndexTTS2 virtualenv dependencies (#5201) by @OliverBryant in https://github.com/xorbitsai/inference/pull/5223
* fix(vllm): add #system_numpy# and pin vllm to 0.21.0 for Ornith/qwen3.5/qwen3.6 by @m199369309 in https://github.com/xorbitsai/inference/pull/5235
* fix(mlx): keep VLM generation on a dedicated thread by @qinxuye in https://github.com/xorbitsai/inference/pull/5234
* fix: disambiguate replica model UIDs with -rep{n} suffix by @Solaris-star in https://github.com/xorbitsai/inference/pull/5215
* fix: allow virtualenv packages to override system markers by @LauraGPT in https://github.com/xorbitsai/inference/pull/5204
* fix(embedding): re-normalize per-item index after batch split by @m199369309 in https://github.com/xorbitsai/inference/pull/5240
* FIX: suppress GPU warnings in CPU image by @leslie2046 in https://github.com/xorbitsai/inference/pull/5238
* fix(i18n): change SSO button text from Keycloak to generic SSO by @m199369309 in https://github.com/xorbitsai/inference/pull/5244
* bug: cover list_models tolerance for stale replica uids by @OliverBryant in https://github.com/xorbitsai/inference/pull/5168
* fix: stabilize Jina v5 and Qwen3 reranker deployment by @OliverBryant in https://github.com/xorbitsai/inference/pull/5232
* FIX: worker reconnection after supervisor restart by @leslie2046 in https://github.com/xorbitsai/inference/pull/5250
* fix(webui): improve worker node display in multi-select by @m199369309 in https://github.com/xorbitsai/inference/pull/5255
* fix(worker): skip redundant virtualenv setup when replicas share the same venv by @m199369309 in https://github.com/xorbitsai/inference/pull/5256
* fix(audio): don't reopen a NamedTemporaryFile so TTS works on Windows (#5261) by @Anai-Guo in https://github.com/xorbitsai/inference/pull/5262
## Documentation
* doc: fix v3.0.0 Chinese release links by @qinxuye in https://github.com/xorbitsai/inference/pull/5205
## Others
* ci: use merge-base diff for PR GPU tests by @qinxuye in https://github.com/xorbitsai/inference/pull/5224
* docs: update release notes links from xinference.io to xinference.co by @martinma51 in https://github.com/xorbitsai/inference/pull/5248
* Consolidate packaging configuration into pyproject.toml by @OliverBryant in https://github.com/xorbitsai/inference/pull/5249
* docs(i18n): update ja/ko/pt_BR release notes links to xinference.co by @martinma51 in https://github.com/xorbitsai/inference/pull/5253

## New Contributors
* @Solaris-star made their first contribution in https://github.com/xorbitsai/inference/pull/5215
* @LauraGPT made their first contribution in https://github.com/xorbitsai/inference/pull/5204
* @SuperMarioYL made their first contribution in https://github.com/xorbitsai/inference/pull/5245
* @martinma51 made their first contribution in https://github.com/xorbitsai/inference/pull/5248

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v3.0.0...v3.1.0