Starstruk Logo MarkStarstruk

v3.0.0

xorbitsai/inferencev3.0.0Jul 19, 2026by XprobeBot

AI Summary

Version 3.0.0 introduces a major frontend overhaul by switching the default interface to Next.js and enabling database-backed authentication by default. The release significantly enhances security with new Audit Center and Security Settings pages, while also providing production-ready offline deployment capabilities via Docker Compose. Numerous bug fixes and model updates ensure improved stability and feature support for the inference platform.

Key Highlights

  • Default frontend switched to Next.js, removing legacy React UI components.
  • Database-backed authentication enabled by default with offline password reset support.
  • New Security Settings and Audit Center pages for enhanced management.
  • Production-ready Docker Compose deployment with offline private PyPI profile.
  • Per-model GPU memory display in the Running Models page.

Breaking Changes

  • Legacy React web UI has been removed in favor of the new Next.js frontend.

New Features

  • Switch default frontend to Next.js app
  • API Key, User Management, refreshToken, change password after first login
  • Support BCE embeddings with vLLM
  • Security Settings Page and Audit Center Page
  • Production-ready Docker Compose deployment with offline private PyPI profile
  • Show per-model GPU memory in Running Models
  • Slim GPU Docker image without pre-installed inference engines

Full Release Notes

# What's new in 3.0.0 (2026-07-19)
These are the changes in inference v3.0.0.
## New features
* FEAT: Switch default frontend to Next.js app by @OliverBryant in https://github.com/xorbitsai/inference/pull/5111
* feat: (UI) API Key, User Management, refreshToken, change password after first login by @maoyuehui in https://github.com/xorbitsai/inference/pull/5143
* feat(api): validate system message ordering for strict Qwen3-family chat templates by @m199369309 in https://github.com/xorbitsai/inference/pull/5149
* feat(embedding): support BCE embeddings with vLLM by @amumu96 in https://github.com/xorbitsai/inference/pull/5158
* Feat: (UI) Added Security Settings Page、Audit Center Page by @maoyuehui in https://github.com/xorbitsai/inference/pull/5153
* feat: enable advanced database-backed authentication by default by @OliverBryant in https://github.com/xorbitsai/inference/pull/5144
* FEAT: production-ready Docker Compose deployment with offline private PyPI profile by @OliverBryant in https://github.com/xorbitsai/inference/pull/5174
* feat(auth): consolidate to database-backed auth and add offline password reset by @OliverBryant in https://github.com/xorbitsai/inference/pull/5180
* FEAT(webui): show per-model GPU memory in Running Models by @OliverBryant in https://github.com/xorbitsai/inference/pull/5186
* feat(llama.cpp): install GPU xllamacpp wheel from CUDA-matched index by @OliverBryant in https://github.com/xorbitsai/inference/pull/5185
* FEAT: prebuilt pypiserver mirror image as the offline compose package source by @qinrui777 in https://github.com/xorbitsai/inference/pull/5048
* FEAT: slim GPU Docker image without pre-installed inference engines by @OliverBryant in https://github.com/xorbitsai/inference/pull/5188
## Enhancements
* ENH: update models JSON [audio] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5142
* ENH: update models JSON [embedding] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5159
* ENH: update 2 models JSON ("CogVideoX-2b", "HunyuanVideo") by @yiboyasss in https://github.com/xorbitsai/inference/pull/5164
* ENH: update model "CogVideoX-5b" JSON by @yiboyasss in https://github.com/xorbitsai/inference/pull/5165
* ENH: update models JSON [audio] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5171
* ENH: update models JSON [embedding, rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5192
* ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5197
* ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5199
* bld: Fix GitHub gpu ci by @OliverBryant in https://github.com/xorbitsai/inference/pull/5107
* bld: Narrow GPU CI triggers and drop always-skipped tests by @OliverBryant in https://github.com/xorbitsai/inference/pull/5182
## Bug fixes
* fix(auth): advanced-auth live-read via DB-current permissions by @m199369309 in https://github.com/xorbitsai/inference/pull/5135
* fix(vllm): flashinfer AOT post-install hook for sm_120 Blackwell by @m199369309 in https://github.com/xorbitsai/inference/pull/5137
* fix(audio): bump funasr pin for Fun-ASR-Nano models to ~=1.3 by @m199369309 in https://github.com/xorbitsai/inference/pull/5140
* fix(frontend): route params via usePathname(); remove Gradio, add LaTeX rendering by @OliverBryant in https://github.com/xorbitsai/inference/pull/5146
* fix: replace Korean text with Japanese in ja locale launchModel keys by @kejhz653-stack in https://github.com/xorbitsai/inference/pull/5154
* fix(embedding): implement truncate_prompt_tokens, OOM eviction, and wait_for_load after recovery by @m199369309 in https://github.com/xorbitsai/inference/pull/5151
* fix(tool_parser): fix deepsee-v4 stream tool calls error by @llyycchhee in https://github.com/xorbitsai/inference/pull/5155
* fix(worker): use os.replace for atomic launch_args persist on Windows (#5161) by @Anai-Guo in https://github.com/xorbitsai/inference/pull/5162
* fix(supervisor): resolve comma-separated worker_ip in sharded launch (#5138) by @Anai-Guo in https://github.com/xorbitsai/inference/pull/5160
* fix(llama.cpp): resolve mmproj against the model directory so image input works by @Anai-Guo in https://github.com/xorbitsai/inference/pull/5170
* fix(oauth2): revoke refresh tokens on password reset by @OliverBryant in https://github.com/xorbitsai/inference/pull/5177
* Fix: (UI) copyToClipboard 、fix the locales Running Model Page by @maoyuehui in https://github.com/xorbitsai/inference/pull/5175
* fix(core): free GPU memory when stopping pytorch rerank models by @OliverBryant in https://github.com/xorbitsai/inference/pull/5173
* BUG: sglang venv inherits incompatible parent transformers under skip_installed by @OliverBryant in https://github.com/xorbitsai/inference/pull/5181
* fix: honor virtualenv exemption in engine matching for multimodal LLM, embedding and rerank by @OliverBryant in https://github.com/xorbitsai/inference/pull/5179
* fix(image/ocr): import torchvision lazily in deepseek_ocr (#5189) by @Anai-Guo in https://github.com/xorbitsai/inference/pull/5191
* fix(worker): create model subpool after virtualenv is installed by @kejhz653-stack in https://github.com/xorbitsai/inference/pull/5196
* fix: pass torch version to offline package builds by @qinxuye in https://github.com/xorbitsai/inference/pull/5200
* fix: mirror dependencies declared by wheels built from sdists by @OliverBryant in https://github.com/xorbitsai/inference/pull/5202
## Documentation
* doc: update v2.12.0 release docs by @qinxuye in https://github.com/xorbitsai/inference/pull/5136
* doc: xinference release notes links by @OliverBryant in https://github.com/xorbitsai/inference/pull/5139
* DOC: document Xinference 3.0 migration and breaking changes by @OliverBryant in https://github.com/xorbitsai/inference/pull/5194
## Others
* ci: narrow GPU test group detection to paths that actually affect GPU tests by @OliverBryant in https://github.com/xorbitsai/inference/pull/5148
* refactor: move XINFERENCE_MODEL_ACTOR_AUTO_RECOVER_LIMIT to constants.py by @m199369309 in https://github.com/xorbitsai/inference/pull/5152
* ci: fix Metal CI failures with latest MLX on macOS 26 runners by @OliverBryant in https://github.com/xorbitsai/inference/pull/5125
* docs(monitor): enhance metrics documentation with cluster label, status detail, and HTTP middleware by @m199369309 in https://github.com/xorbitsai/inference/pull/5184
* chore: remove legacy React web UI now that Next.js frontend is the default by @OliverBryant in https://github.com/xorbitsai/inference/pull/5145


**Full Changelog**: https://github.com/xorbitsai/inference/compare/v2.12.0...v3.0.0