Starstruk Logo MarkStarstruk

v3.2.0

xorbitsai/inferencev3.2.0Aug 15, 2026by XprobeBot

AI Summary

This release introduces extensive support for new models across various modalities including LLMs, embeddings, rerankers, audio, video, and image generation. It also adds significant infrastructure improvements like per-replica worker and GPU placement, alongside enhancements to the WebUI and OCR capabilities.

Key Highlights

  • Support for new models such as R3-embedding-0.6b, R3-rerank-0.6b, jina-reranker-v3.5, minimax-m3, Ling 3.0, and Wan2.2 Animate 2.
  • Introduction of per-replica worker and GPU placement features with WebUI controls.
  • Enhanced OCR capabilities including DeepDoc (ragflow) support and PDF input handling.
  • Logprobs support added to chat and completions endpoints.
  • New audio models including indextts 2.5 and CAMPPlus speaker embedding.

New Features

  • Surface logprobs in /v1/chat/completions and /v1/completions.
  • Support deploying R3-embedding-0.6b and R3-rerank-0.6b models.
  • Support jina-reranker-v3.5 with its native listwise API.
  • PDF input support for the OCR API.
  • Add DeepDoc (ragflow) OCR model as builtin image model.
  • Add per-replica worker and GPU placement.
  • Add per-replica placement controls in WebUI.
  • Add running-model replica scaling in WebUI.
  • Expose DeepDoc's document-parsing pipeline as task='parse'.
  • Add CAMPPlus speaker embedding support.
  • Add FireRed Image Edit 1.1 support.
  • Add Wan2.2 Animate 2 video models.
  • Auto-detect model download hub between Hugging Face and ModelScope.
  • Add Ling 3.0 model support.
  • Add indextts 2.5 model.
  • Add OvisOCR2 model support.
  • Enhance download progress details.

Full Release Notes

# What's new in 3.2.0 (2026-08-15)
These are the changes in inference v3.2.0.
## New features
* feat(llm): surface logprobs in /v1/chat/completions by @SuperMarioYL in https://github.com/xorbitsai/inference/pull/5252
* feat: support deploying R3-embedding-0.6b and R3-rerank-0.6b models by @xiaoyesoso in https://github.com/xorbitsai/inference/pull/5272
* feat(rerank): support jina-reranker-v3.5 with its native listwise API by @xiaoyesoso in https://github.com/xorbitsai/inference/pull/5269
* feat: PDF input support for the OCR API by @OliverBryant in https://github.com/xorbitsai/inference/pull/5280
* feat(sglang): surface logprobs in /v1/completions and /v1/chat/completions by @SuperMarioYL in https://github.com/xorbitsai/inference/pull/5270
* feat: add DeepDoc (ragflow) OCR model as builtin image model by @kejhz653-stack in https://github.com/xorbitsai/inference/pull/5230
* feat(model):support minimax-m3 model by @llyycchhee in https://github.com/xorbitsai/inference/pull/5273
* feat: support per-replica worker and GPU placement by @m199369309 in https://github.com/xorbitsai/inference/pull/5286
* feat(webui): add per-replica placement controls by @m199369309 in https://github.com/xorbitsai/inference/pull/5287
* feat: support scaling running models by replica by @m199369309 in https://github.com/xorbitsai/inference/pull/5288
* feat(webui): add running-model replica scaling by @m199369309 in https://github.com/xorbitsai/inference/pull/5289
* feat: expose and display per-replica runtime resources by @m199369309 in https://github.com/xorbitsai/inference/pull/5291
* feat(ocr): expose DeepDoc's document-parsing pipeline as task="parse" by @OliverBryant in https://github.com/xorbitsai/inference/pull/5299
* feat(audio): add CAMPPlus speaker embedding support by @leslie2046 in https://github.com/xorbitsai/inference/pull/5298
* feat(image): add FireRed Image Edit 1.1 support by @Minamiyama in https://github.com/xorbitsai/inference/pull/5306
* feat: add Wan2.2 Animate 2 video models by @Minamiyama in https://github.com/xorbitsai/inference/pull/5309
* feat: auto-detect model download hub between Hugging Face and ModelScope by @OliverBryant in https://github.com/xorbitsai/inference/pull/5265
* feat(video): add MiniMax-H3 support by @qinxuye in https://github.com/xorbitsai/inference/pull/5321
* feat(llm): add Ling 3.0 model support by @Minamiyama in https://github.com/xorbitsai/inference/pull/5311
* Feat: enhance download progress details by @Minamiyama in https://github.com/xorbitsai/inference/pull/5310
* feat: [New Model] indextts 2.5 by @Minamiyama in https://github.com/xorbitsai/inference/pull/5319
* feat: [New Model] Add OvisOCR2 model support by @Minamiyama in https://github.com/xorbitsai/inference/pull/5322
## Enhancements
* ENH: update models JSON [embedding, rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5276
* ENH: update models JSON [rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5279
* ENH: update models JSON [image, llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5292
* ENH: update models JSON [image] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5294
* ENH: add searchable dropdowns to Audit Center text filters by @leslie2046 in https://github.com/xorbitsai/inference/pull/5282
* ENH: add page navigation and deep pagination to log and audit centers by @leslie2046 in https://github.com/xorbitsai/inference/pull/5304
* ENH: update models JSON [image, llm, video] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5316
* ENH: update models JSON [audio, llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/5324
* perf(supervisor): add whole-result debounce cache for list_models by @m199369309 in https://github.com/xorbitsai/inference/pull/5264
## Bug fixes
* fix(model): avoid Set-changed-during-iteration crash and gc.get_objects() overhead under concurrent downloads by @SuperMarioYL in https://github.com/xorbitsai/inference/pull/5257
* fix: add anthropic SDK to core dependencies for route registration by @m199369309 in https://github.com/xorbitsai/inference/pull/5281
* FIX: use Xorbits/deepdoc mirror for the DeepDoc huggingface source by @OliverBryant in https://github.com/xorbitsai/inference/pull/5293
* fix(webui): ignore stale launch model requests by @m199369309 in https://github.com/xorbitsai/inference/pull/5285
* BUG: fix audit center text filters returning no results by @OliverBryant in https://github.com/xorbitsai/inference/pull/5277
* fix(worker): preserve Jina v3 allocator env for recovery by @m199369309 in https://github.com/xorbitsai/inference/pull/5300
* fix: preserve replicas across worker recovery by @m199369309 in https://github.com/xorbitsai/inference/pull/5290
* fix(ocr): stop task="parse" rejecting ordinary multi-page PDFs by @OliverBryant in https://github.com/xorbitsai/inference/pull/5308
* fix(vllm): keep WorkerActor importable without vllm for multi-GPU launch by @bluefish-08 in https://github.com/xorbitsai/inference/pull/5312
* fix(deploy): stop polling requests flooding the access log by @bluefish-08 in https://github.com/xorbitsai/inference/pull/5317
* fix(llm): declare audioread for qwen_omni_utils virtualenvs by @bluefish-08 in https://github.com/xorbitsai/inference/pull/5313
* fix: make cancelling a model launch actually cancel it by @bluefish-08 in https://github.com/xorbitsai/inference/pull/5315
* fix: normalize replica during autostart recovery by @medisean in https://github.com/xorbitsai/inference/pull/5318
* fix: repair pypiserver release dependencies by @qinxuye in https://github.com/xorbitsai/inference/pull/5326
* fix: mirror sdists with broken metadata builds by @qinxuye in https://github.com/xorbitsai/inference/pull/5329
## Documentation
* doc: update README models and links for v3.1.0 by @qinxuye in https://github.com/xorbitsai/inference/pull/5267
## Others
* docs: add v3.1.0 to release notes index by @martinma51 in https://github.com/xorbitsai/inference/pull/5274
* docs: fix broken star history chart links by @SimonFoobar648 in https://github.com/xorbitsai/inference/pull/5323
* docs: update README new model highlights by @qinxuye in https://github.com/xorbitsai/inference/pull/5325

## New Contributors
* @SimonFoobar648 made their first contribution in https://github.com/xorbitsai/inference/pull/5323
* @medisean made their first contribution in https://github.com/xorbitsai/inference/pull/5318

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v3.1.0...v3.2.0