Starstruk Logo MarkStarstruk

v3.3.0

xorbitsai/inferencev3.3.0Aug 30, 2026by XprobeBot

AI Summary

This release introduces extensive new features including support for multiple new AI models (Qwen3.8, DeepSeek V4, Ornith-1.5, Kimi-K3, etc.), new audio and image generation models, enhanced router capabilities with token-aware routing and agent-based orchestration, and various UI enhancements.

Key Highlights

  • Support for Qwen3.8-27B and MiniMax music3 models
  • Token-aware virtual model routing with Agent-based orchestration
  • New audio models including ACE-Step 1.5 and FireRedTTS3
  • New image models including Ideogram4, HiDream-O1, and GLM-Image
  • Enhanced WebUI with IndexTTS emotion controls and media seed controls

New Features

  • Support for jina-reranker-m0
  • Support for qwen3.8-27B and complete Qwen3.8 support
  • Support for MiniMax-H3 Lightning LoRA
  • Support for Ornith-1.5-35B-A3B and Ornith-1.5-397B
  • Support for Kimi-K3
  • Support for DeepSeek V4 Flash 0731
  • Support for ACE-Step 1.5 music generation
  • Support for Krea 2 models
  • Support for world model generation
  • Support for replica config in launch CLI and Web UI
  • Support for configurable dynamic replica scale-up
  • Support for WeMM-Embedding
  • Support for Breeze-TTS-2
  • Support for NaviDC-OCR
  • Support for secure per-launch find-links support
  • Add unified Anthropic Messages protocol adapter
  • Add Token Router management
  • Add Agent-based Router orchestration
  • Add Token Router observability
  • Add Router Agent and cluster management
  • Add persistent system settings management

Full Release Notes

# What's new in 3.3.0 (2026-08-30)
These are the changes in inference v3.3.0.
## New features
* feat(rerank): support jina-reranker-m0 by @Minamiyama in https://github.com/xorbitsai/inference/pull/5327
* feat(webui): show downloaded file sizes by @Minamiyama in https://github.com/xorbitsai/inference/pull/5330
* feat(model): support qwen3.8-27B by @bluefish-08 in https://github.com/xorbitsai/inference/pull/5337
* feat(webui): add IndexTTS emotion controls by @Minamiyama in https://github.com/xorbitsai/inference/pull/5335
* feat(audio): support multiple engines and MLX models by @qinxuye in https://github.com/xorbitsai/inference/pull/5334
* feat: [new model] minimax music3 by @Minamiyama in https://github.com/xorbitsai/inference/pull/5345
* feat(model): complete Qwen3.8 support by @qinxuye in https://github.com/xorbitsai/inference/pull/5339
* feat: support MiniMax-H3 Lightning LoRA by @qinxuye in https://github.com/xorbitsai/inference/pull/5338
* feat(audio): support FireRedTTS3 by @Minamiyama in https://github.com/xorbitsai/inference/pull/5352
* feat(llm): register DeepSeek V4 Flash 0731 by @m199369309 in https://github.com/xorbitsai/inference/pull/5371
* feat(image): add Ideogram4 support by @Minamiyama in https://github.com/xorbitsai/inference/pull/5367
* feat(image): sensenova u1.5 by @Minamiyama in https://github.com/xorbitsai/inference/pull/5369
* feat(image): support HiDream-O1 models by @Minamiyama in https://github.com/xorbitsai/inference/pull/5370
* feat(image): add GLM-Image support by @Minamiyama in https://github.com/xorbitsai/inference/pull/5394
* feat(router): add token-aware virtual model routing by @m199369309 in https://github.com/xorbitsai/inference/pull/5375
* feat(image): update SenseNova U1.5 model info by @Minamiyama in https://github.com/xorbitsai/inference/pull/5401
* feat(router): add tokenizer asset registry by @m199369309 in https://github.com/xorbitsai/inference/pull/5376
* feat: add media seed controls and dimension swap by @Minamiyama in https://github.com/xorbitsai/inference/pull/5400
* feat(router): bundle DeepSeek V4 tokenizer asset by @m199369309 in https://github.com/xorbitsai/inference/pull/5377
* feat(ui): add Token Router management by @m199369309 in https://github.com/xorbitsai/inference/pull/5378
* feat(router): add Agent-based Router orchestration by @m199369309 in https://github.com/xorbitsai/inference/pull/5379
* feat(llm): support Ornith-1.5-35B-A3B by @m199369309 in https://github.com/xorbitsai/inference/pull/5406
* feat(monitoring): add Token Router observability by @m199369309 in https://github.com/xorbitsai/inference/pull/5381
* feat(ui): add Router Agent and cluster management by @m199369309 in https://github.com/xorbitsai/inference/pull/5380
* feat(api): add unified Anthropic Messages protocol adapter by @m199369309 in https://github.com/xorbitsai/inference/pull/5389
* FEAT: [model] glm-5.2 support by @llyycchhee in https://github.com/xorbitsai/inference/pull/5404
* feat(venv): add secure per-launch find-links support by @m199369309 in https://github.com/xorbitsai/inference/pull/5387
* feat(router): route Anthropic Messages through virtual models by @m199369309 in https://github.com/xorbitsai/inference/pull/5391
* feat(router): expose virtual model running details by @m199369309 in https://github.com/xorbitsai/inference/pull/5397
* feat(venv): install Jina flash-attn wheel from find-links by @m199369309 in https://github.com/xorbitsai/inference/pull/5392
* feat(ui): add per-launch virtualenv find-links by @m199369309 in https://github.com/xorbitsai/inference/pull/5388
* feat(llm): support Ornith-1.5-397B by @m199369309 in https://github.com/xorbitsai/inference/pull/5405
* feat(audio): add ACE-Step 1.5 music generation support by @Minamiyama in https://github.com/xorbitsai/inference/pull/5413
* feat(image): add Krea 2 models by @Minamiyama in https://github.com/xorbitsai/inference/pull/5419
* FEAT: add world model generation support by @qinxuye in https://github.com/xorbitsai/inference/pull/5414
* FEAT: support replica config in launch CLI and Web UI by @m199369309 in https://github.com/xorbitsai/inference/pull/5422
* FEAT: [model] Kimi-K3 support by @llyycchhee in https://github.com/xorbitsai/inference/pull/5417
* FEAT: support configurable dynamic replica scale-up by @leslie2046 in https://github.com/xorbitsai/inference/pull/5426
* feat: add persistent system settings management by @Minamiyama in https://github.com/xorbitsai/inference/pull/5427
* FEAT: support NaviDC-OCR with Transformers and vLLM by @Minamiyama in https://github.com/xorbitsai/inference/pull/5431
* feat(embedding): add WeMM-Embedding support by @Minamiyama in https://github.com/xorbitsai/inference/pull/5439
* FEAT(model): Add Breeze-TTS-2 support by @Minamiyama in https://github.com/xorbitsai/inference/pull/5437
* feat(ui): show full paths on hover by @Minamiyama in https://github.com/xorbitsai/inference/pull/5444
## Enhancements
* ENH: update models JSON [rerank, video] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5328
* ENH(ui): make image previews adaptive by @Minamiyama in https://github.com/xorbitsai/inference/pull/5385
* Enh: Expose VoiceDesign ability to Qwen3-TTS-Voice-Design by @Minamiyama in https://github.com/xorbitsai/inference/pull/5355
* ENH: Add multi-engine video support with MLX by @qinxuye in https://github.com/xorbitsai/inference/pull/5407
* ENH(UI): Add configurable speech and music output formats by @Minamiyama in https://github.com/xorbitsai/inference/pull/5436
* ENH: support Base64 video input in vLLM chat by @amumu96 in https://github.com/xorbitsai/inference/pull/5434
## Bug fixes
* fix(audio): correct IndexTTS-2.5 runtime dependencies by @Minamiyama in https://github.com/xorbitsai/inference/pull/5331
* fix(audio): return playable speech responses by @Minamiyama in https://github.com/xorbitsai/inference/pull/5332
* BUG: pin DeepDoc transformers to >=4.51,<5 for OCR launches by @1084669403 in https://github.com/xorbitsai/inference/pull/5340
* fix(ui): auto-size launch model dropdowns by @Minamiyama in https://github.com/xorbitsai/inference/pull/5343
* fix(core): guard missing/invalid usage in non-stream chat metrics by @Hyhyhyyy in https://github.com/xorbitsai/inference/pull/5342
* fix(model): calculate download progress per file by @Minamiyama in https://github.com/xorbitsai/inference/pull/5346
* fix(ui): include scrollbar in dropdown width by @Minamiyama in https://github.com/xorbitsai/inference/pull/5354
* fix: validate LLM metadata before packaging by @li2631026381-alt in https://github.com/xorbitsai/inference/pull/5353
* fix(core): refine autostart transition handling and reset retry attempts by @baka-world in https://github.com/xorbitsai/inference/pull/5361
* fix(worker): wait reliably for metrics exporter startup by @baka-world in https://github.com/xorbitsai/inference/pull/5363
* fix(core): make worker supervisor reconnection generation-safe by @baka-world in https://github.com/xorbitsai/inference/pull/5364
* fix(model): harden batch processor against cancellation and unexpected exits by @baka-world in https://github.com/xorbitsai/inference/pull/5362
* fix(core): normalize distributed worker count by @baka-world in https://github.com/xorbitsai/inference/pull/5366
* fix(llm): preserve DeepSeek V4 tool argument types by @m199369309 in https://github.com/xorbitsai/inference/pull/5372
* fix(vllm): handle engine death during async iteration by @m199369309 in https://github.com/xorbitsai/inference/pull/5374
* fix(vllm): use block size 256 for DeepSeek V4 by @m199369309 in https://github.com/xorbitsai/inference/pull/5373
* fix(api): map Qwen3.8 top-level reasoning effort by @baka-world in https://github.com/xorbitsai/inference/pull/5383
* fix(image): pin Qwen Layered dependencies by @Minamiyama in https://github.com/xorbitsai/inference/pull/5390
* fix(cache): fully remove downloaded models by @Minamiyama in https://github.com/xorbitsai/inference/pull/5395
* fix(supervisor): reschedule autostart when the last replica dies by @AmirF194 in https://github.com/xorbitsai/inference/pull/5409
* fix(venv): honor configured sources in post-install hooks by @m199369309 in https://github.com/xorbitsai/inference/pull/5386
* fix(ui): clarify Token Router identifiers and asset layout by @m199369309 in https://github.com/xorbitsai/inference/pull/5393
* fix(router): separate runtime and backend credentials by @m199369309 in https://github.com/xorbitsai/inference/pull/5396
* fix(router): stabilize managed runtime instance identity by @m199369309 in https://github.com/xorbitsai/inference/pull/5398
* fix(ui): harden Token Router running model and detail views by @m199369309 in https://github.com/xorbitsai/inference/pull/5399
* fix(llm): support Hugging Face cache roots in auto-fill by @amumu96 in https://github.com/xorbitsai/inference/pull/5403
* FIX: stabilize MLX runtime dependencies and locks by @qinxuye in https://github.com/xorbitsai/inference/pull/5423
* fix: declare Token Router installation extra by @m199369309 in https://github.com/xorbitsai/inference/pull/5424
* fix(supervisor): reschedule autostart on whole-worker death by @AmirF194 in https://github.com/xorbitsai/inference/pull/5429
* fix: skip cache status update when cache tracker ref is None by @BetterAndBetterII in https://github.com/xorbitsai/inference/pull/5443
* FIX: preserve available model download hubs by @Minamiyama in https://github.com/xorbitsai/inference/pull/5438
* fix(image): classify OvisOCR2 as OCR by @Minamiyama in https://github.com/xorbitsai/inference/pull/5435
* FIX: stabilize HY-WorldPlay runtime and progress reporting by @qinxuye in https://github.com/xorbitsai/inference/pull/5432
* FIX: restore GPU and aarch64 Docker builds by @qinxuye in https://github.com/xorbitsai/inference/pull/5451
* FIX: repair pypiserver platform dependencies by @qinxuye in https://github.com/xorbitsai/inference/pull/5453
## Others
* docs: add v3.2.0 to release notes index and locale catalogs by @martinma51 in https://github.com/xorbitsai/inference/pull/5348
* ci: skip GPU tests for video and flexible model changes by @OliverBryant in https://github.com/xorbitsai/inference/pull/5368
* refactor(router): remove legacy Agent compatibility by @m199369309 in https://github.com/xorbitsai/inference/pull/5421
* chore: refresh built-in model catalog and skip draft CI by @qinxuye in https://github.com/xorbitsai/inference/pull/5411

## New Contributors
* @1084669403 made their first contribution in https://github.com/xorbitsai/inference/pull/5340
* @Hyhyhyyy made their first contribution in https://github.com/xorbitsai/inference/pull/5342
* @li2631026381-alt made their first contribution in https://github.com/xorbitsai/inference/pull/5353
* @baka-world made their first contribution in https://github.com/xorbitsai/inference/pull/5361
* @AmirF194 made their first contribution in https://github.com/xorbitsai/inference/pull/5409
* @BetterAndBetterII made their first contribution in https://github.com/xorbitsai/inference/pull/5443

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v3.2.0...v3.3.0