v3.5.0 v4.0.0
xorbitsai/inferenceSep 25, 2026 → Oct 10, 202614 days between releases
What's New
Major version 4.0.0 adds Xavier KV cache sharing, cross-engine deployments, and new API key requirements.
Breaking Changes
- Globally unique API key names requirement
- Cross-engine prefill/decode separation changes infrastructure
New Features
- Xavier KV cache sharing across vLLM, SGLang, and MLX
- Stateless OpenAI Responses API (/v1/responses)
- Jina OCR v1 support
- Confucius4-TTS support
- Runtime Log Center
Improvements
- Native NIXL backend for vLLM and SGLang PD deployments
- GPU weight caching enables reconfiguration without reloading weights
- Distributed vLLM V1 inference with improved scheduling
- Cross-platform installation and local service management
- Advanced Xavier KV cache sharing to reuse context
Bug Fixes
- Bug fixes for cross-platform compatibility
- Improved pipeline scheduling and failure handling
Upgrade Notes
API key names must now be globally unique. Review configurations for cross-engine prefill/decode separation. Installation now uses a one-command method for Linux, macOS, and Windows.
The Two Releases
v3.5.0
Sep 25, 2026· 26 days ago
This release addresses a security vulnerability regarding custom-model registration and introduces extensive new features including support for various audio, image, and LLM models, along with logging and monitoring capabilities.
Full release notesv4.0.0
Oct 10, 2026· 12 days ago
Version 4.0.0 introduces significant performance and architectural improvements, including native NIXL backend support for vLLM and SGLang PD deployments, advanced Xavier KV cache sharing across multiple engines, and a new runtime log center. The release adds new capabilities like Jina OCR v1, Confucius4-TTS, and a Stateless OpenAI Responses API, alongside performance optimizations and bug fixes for cross-platform compatibility.
Full release notes