v3.4.0

# What's new in 3.4.0 (2026-09-11)
These are the changes in inference v3.4.0.
## New features
* feat(vllm): support native multiprocessing executor routing by @m199369309 in https://github.com/xorbitsai/inference/pull/5447
* feat: Add Gemma-4 Transformers backend with batching support by @maoyuehui in https://github.com/xorbitsai/inference/pull/5430
* feat(model): add download-only flow by @Minamiyama in https://github.com/xorbitsai/inference/pull/5463
* feat(model): support dots ocr model by @llyycchhee in https://github.com/xorbitsai/inference/pull/5468
* feat(model): add engine registration hooks by @amumu96 in https://github.com/xorbitsai/inference/pull/5465
* feat(cache): add resumable cache management by @Minamiyama in https://github.com/xorbitsai/inference/pull/5471
* feat(cache): open cached models for launch by @Minamiyama in https://github.com/xorbitsai/inference/pull/5476
* feat(ui): select sole available model engine by @Minamiyama in https://github.com/xorbitsai/inference/pull/5480
* FEAT: support oci:// model URIs via llmman serve by @ericcurtin in https://github.com/xorbitsai/inference/pull/5457
* feat(model): support MonkeyOCR by @llyycchhee in https://github.com/xorbitsai/inference/pull/5475
* feat(model): Add Fish Audio S1-mini and S2-Pro support by @Minamiyama in https://github.com/xorbitsai/inference/pull/5490
* feat: support multi-worker replica scaling by @leslie2046 in https://github.com/xorbitsai/inference/pull/5491
* feat(models): add embedding and rerank multimodal abilities by @Minamiyama in https://github.com/xorbitsai/inference/pull/5496
* feat(ui): prefer system download source by @Minamiyama in https://github.com/xorbitsai/inference/pull/5498
* feat: support prefill/decode separation by @amumu96 in https://github.com/xorbitsai/inference/pull/5505
* feat(audio): add multilingual TTS language controls by @Minamiyama in https://github.com/xorbitsai/inference/pull/5500
* feat(i18n): add zh-TW (Taiwan Traditional Chinese) locale by @leonoxo in https://github.com/xorbitsai/inference/pull/5509
* feat(llm): add MiniCPM5-2B support by @Minamiyama in https://github.com/xorbitsai/inference/pull/5506
* feat(webui): reuse launch history across models by @m199369309 in https://github.com/xorbitsai/inference/pull/5517
## Enhancements
* ENH(auth): Add granular system settings permissions by @Minamiyama in https://github.com/xorbitsai/inference/pull/5445
* ENH: support JoyAI image editing models by @Minamiyama in https://github.com/xorbitsai/inference/pull/5458
* ENH: Add TTS streaming playback to Web UI by @Minamiyama in https://github.com/xorbitsai/inference/pull/5487
* ENH: Complete SD WebUI image workflow compatibility by @amumu96 in https://github.com/xorbitsai/inference/pull/5508
* ENH: update frontend language order and product links by @qinxuye in https://github.com/xorbitsai/inference/pull/5514
* BLD: support Python 3.14 and require xoscar 0.10.0 by @qinxuye in https://github.com/xorbitsai/inference/pull/5515
* BLD: fix CPU Docker image and add non-publishing manual builds by @qinxuye in https://github.com/xorbitsai/inference/pull/5524
## Bug fixes
* fix(venv): require xoscar native find-links support by @m199369309 in https://github.com/xorbitsai/inference/pull/5448
* fix(embedding): align Jina flash-attn version by @m199369309 in https://github.com/xorbitsai/inference/pull/5452
* fix(venv): protect virtual environments in use by @m199369309 in https://github.com/xorbitsai/inference/pull/5449
* fix(llm): emit streaming tool call metadata once by @m199369309 in https://github.com/xorbitsai/inference/pull/5446
* FIX: Restore supported vLLM and SGLang model engines by @qinxuye in https://github.com/xorbitsai/inference/pull/5461
* FIX: apply download worker environment to snapshots by @amumu96 in https://github.com/xorbitsai/inference/pull/5460
* fix(llm): gate Hy-MT2 remote code execution by @m199369309 in https://github.com/xorbitsai/inference/pull/5474
* fix(transformers): support legacy cache conversion by @m199369309 in https://github.com/xorbitsai/inference/pull/5477
* fix(scheduler): isolate batch inference failures by @m199369309 in https://github.com/xorbitsai/inference/pull/5478
* fix(transformers): handle multi-device cache reduction by @m199369309 in https://github.com/xorbitsai/inference/pull/5479
* fix(security): report bans lifted, not records dropped, from bulk unban by @NgoQuocViet2001 in https://github.com/xorbitsai/inference/pull/5483
* fix(audio): correct F5-TTS logging format by @David-Wu1119 in https://github.com/xorbitsai/inference/pull/5485
* BUG: Merge engine capabilities across workers by @oodadoudou in https://github.com/xorbitsai/inference/pull/5486
* FIX: Restore audio streaming with TorchCodec by @Minamiyama in https://github.com/xorbitsai/inference/pull/5488
* fix(model): make register_builtin_model idempotent across refreshes by @AmirF194 in https://github.com/xorbitsai/inference/pull/5481
* FIX: Improve model experience actions and latency feedback by @leslie2046 in https://github.com/xorbitsai/inference/pull/5492
* fix(ui): contain embedding result scrolling by @Minamiyama in https://github.com/xorbitsai/inference/pull/5495
* fix: handle runtime and device options in flexible Transformers launcher by @leslie2046 in https://github.com/xorbitsai/inference/pull/5497
* fix(rerank): launch GGUF rerank models without explicit engine/quantization by @7487 in https://github.com/xorbitsai/inference/pull/5503
* fix(chatglm): emit a usage-only final streaming chunk by @m199369309 in https://github.com/xorbitsai/inference/pull/5510
* fix(transformers): propagate batched generate request IDs by @m199369309 in https://github.com/xorbitsai/inference/pull/5513
* fix(transformers): handle batched logits when selecting next token by @m199369309 in https://github.com/xorbitsai/inference/pull/5511
* fix(transformers): enforce token limits across batched decode steps by @m199369309 in https://github.com/xorbitsai/inference/pull/5512
* fix(worker): decouple heartbeat and full status reporting by @m199369309 in https://github.com/xorbitsai/inference/pull/5518
* fix(webui): restore server-backed launch history by @m199369309 in https://github.com/xorbitsai/inference/pull/5516
* fix(core): preserve model GPU memory across transient failures by @m199369309 in https://github.com/xorbitsai/inference/pull/5519
* fix(worker): refresh dynamic model subpool PID ownership by @m199369309 in https://github.com/xorbitsai/inference/pull/5520
* FIX: avoid implicit event loop in async client cleanup by @qinxuye in https://github.com/xorbitsai/inference/pull/5521
## Documentation
* doc: update README model highlights for v3.3.0 by @qinxuye in https://github.com/xorbitsai/inference/pull/5454
* DOC: preserve ACE-Step1.5 details outside generated docs by @qinxuye in https://github.com/xorbitsai/inference/pull/5455
* DOC: fix model type card layout by @qinxuye in https://github.com/xorbitsai/inference/pull/5489
## Others
* test(pypiserver): align Jina flash-attn manifest pin by @m199369309 in https://github.com/xorbitsai/inference/pull/5456
* docs: add v3.3.0 to release notes index and locale catalogs by @martinma51 in https://github.com/xorbitsai/inference/pull/5462
* chore: bump actions/checkout to v7 by @qinrui777 in https://github.com/xorbitsai/inference/pull/5469

## New Contributors
* @NgoQuocViet2001 made their first contribution in https://github.com/xorbitsai/inference/pull/5483
* @ericcurtin made their first contribution in https://github.com/xorbitsai/inference/pull/5457
* @David-Wu1119 made their first contribution in https://github.com/xorbitsai/inference/pull/5485
* @oodadoudou made their first contribution in https://github.com/xorbitsai/inference/pull/5486
* @7487 made their first contribution in https://github.com/xorbitsai/inference/pull/5503
* @leonoxo made their first contribution in https://github.com/xorbitsai/inference/pull/5509

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v3.3.0...v3.4.0