Starstruk Logo MarkStarstruk

v2.10.0

xorbitsai/inferencev2.10.0Jun 5, 2026by XprobeBot

AI Summary

A major release focusing on security with OIDC/Keycloak SSO, IP/Key banning, and comprehensive audit logging. It also introduces a revamped logging system using JSON format and a new UI with menu reorganization.

Key Highlights

  • Security: OIDC/Keycloak SSO, IP/Key ban, and rate limiting
  • Audit: Comprehensive audit logging system
  • Logging: JSON format and stdout redirect
  • UI: Menu reorganization and new launch model list
  • Monitor: Per-model GPU memory usage metrics

New Features

  • OIDC/Keycloak SSO authentication support
  • IP/Key ban and rate limiting
  • Audit logging system
  • Logging: JSON format and stdout redirect
  • UI: Menu reorganization and i18n updates
  • Persist launch model configuration history
  • Monitor: Per-model GPU memory usage metrics

Full Release Notes

# What's new in 2.10.0 (2026-06-05)
These are the changes in inference v2.10.0.
## New features
* feat(logging): enhance logging system with JSON format and stdout redirect by @m199369309 in https://github.com/xorbitsai/inference/pull/4947
* feat(auth): add OIDC/Keycloak SSO authentication support by @m199369309 in https://github.com/xorbitsai/inference/pull/4948
* feat(audit): add comprehensive audit logging system by @m199369309 in https://github.com/xorbitsai/inference/pull/4951
* feat(security): add IP/Key ban and rate limiting for brute-force protection by @m199369309 in https://github.com/xorbitsai/inference/pull/4949
* feat(apikey): add description field and ban status display by @m199369309 in https://github.com/xorbitsai/inference/pull/4952
* feat(monitor): add security audit panels and filebeat configurations by @m199369309 in https://github.com/xorbitsai/inference/pull/4953
* feat(ui): menu reorganization, fetchWrapper auth, and i18n updates by @m199369309 in https://github.com/xorbitsai/inference/pull/4954
* feat(monitor): add per-model GPU memory usage metrics by @m199369309 in https://github.com/xorbitsai/inference/pull/4965
* feat(monitor): update Grafana dashboards with GPU memory panels by @m199369309 in https://github.com/xorbitsai/inference/pull/4969
* feat: persist launch model configuration history server-side by @m199369309 in https://github.com/xorbitsai/inference/pull/4972
* FEAT: [UI] update sidebar, login logo and favicon by @yiboyasss in https://github.com/xorbitsai/inference/pull/4978
* feat:  new ui (register json view, formInstance, launch model list, cache/env …) by @maoyuehui in https://github.com/xorbitsai/inference/pull/4966
* feat(logging): add three-level download progress logging by @m199369309 in https://github.com/xorbitsai/inference/pull/4989
* feat(ui): allow editing API key name and description in edit dialog by @m199369309 in https://github.com/xorbitsai/inference/pull/4991
## Enhancements
* ENH: update model "qwen3.6" JSON by @llyycchhee in https://github.com/xorbitsai/inference/pull/4945
* ENH: update model "ChatTTS" JSON by @llyycchhee in https://github.com/xorbitsai/inference/pull/4961
* ENH: update models JSON [embedding] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4971
* ENH: update model "qwen3.6" JSON by @llyycchhee in https://github.com/xorbitsai/inference/pull/4994
## Bug fixes
* fix(vllm): set quantization="fp8" when model_format is fp8 by @m199369309 in https://github.com/xorbitsai/inference/pull/4959
* fix(auth): return specific error messages for expired/disabled API keys by @m199369309 in https://github.com/xorbitsai/inference/pull/4963
* fix(monitor): periodic refresh for security gauges and ban remaining API by @m199369309 in https://github.com/xorbitsai/inference/pull/4964
* BUG: fix jina-embeddings-v2-base-zh deployment dependencies by @m199369309 in https://github.com/xorbitsai/inference/pull/4970
* fix(monitor): capture vLLM/SGLang GPU workers via deferred PID tattoo by @m199369309 in https://github.com/xorbitsai/inference/pull/4977
* fix(vllm): remove best_of for v0.21.0 by @llyycchhee in https://github.com/xorbitsai/inference/pull/4979
* bug: Adapt vLLM LoRA request path parameter by @amumu96 in https://github.com/xorbitsai/inference/pull/4980
* fix(logging): strip all CSI escapes and route flush() through sampling by @m199369309 in https://github.com/xorbitsai/inference/pull/4983
* fix(vllm): read json_schema from `schema_` so guided decoding applies by @m199369309 in https://github.com/xorbitsai/inference/pull/4985
* bug: fix llama.cpp streaming tool call edge cases by @qinxuye in https://github.com/xorbitsai/inference/pull/4988
* fix: Fix GPU info probe on GB10 / DGX Spark (NVML v2 memory-info fallback) by @tbraun96 in https://github.com/xorbitsai/inference/pull/4990
* fix(worker): limit concurrent model launches with semaphore to prevent heartbeat timeouts by @m199369309 in https://github.com/xorbitsai/inference/pull/4992
* fix(venv): allow user override by @llyycchhee in https://github.com/xorbitsai/inference/pull/4993
* fix(venv): evaluate CUDA version markers dynamically by @llyycchhee in https://github.com/xorbitsai/inference/pull/4958
## Documentation
* DOC: add v2.9.0 release notes by @qinxuye in https://github.com/xorbitsai/inference/pull/4941
* doc: Add AI agent guidance for the project by @qinxuye in https://github.com/xorbitsai/inference/pull/4920
* doc: remove WeChat QR links from docs site by @qinxuye in https://github.com/xorbitsai/inference/pull/4967
## Others
* chattts: set weights_only=True for torch.load speaker embedding by @tonghuaroot in https://github.com/xorbitsai/inference/pull/4956

## New Contributors
* @tonghuaroot made their first contribution in https://github.com/xorbitsai/inference/pull/4956
* @tbraun96 made their first contribution in https://github.com/xorbitsai/inference/pull/4990

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v2.9.0...v2.10.0