Starstruk Logo MarkStarstruk

v2.4.0

xorbitsai/inferencev2.4.0Mar 29, 2026by XprobeBot

AI Summary

This release introduces OTEL (OpenTelemetry) support, adds GPU load metrics, upgrades base image to 0.17.1 with aarch64 support, and fixes various issues related to qwen3.5 and MLX backend.

Key Highlights

  • Introducing OTEL
  • Add gpu load metrics
  • Support vLLM 0.18.0
  • Upgrade base image to 0.17.1 with aarch64 support
  • Fix MLX backend accumulation issue

New Features

  • Introducing OTEL
  • Add Xagent link
  • Remove featured/all toggle and prioritize featured models
  • Support vllm v0.18.0
  • Add gpu load metrics
  • Upgrade base image to 0.17.1 and add aarch64 support
  • Fix aarch64 build

Full Release Notes

# What's new in 2.4.0 (2026-03-29)
These are the changes in inference v2.4.0.
## New features
* FEAT: introducing OTEL by @leslie2046 in https://github.com/xorbitsai/inference/pull/4666
* FEAT: [UI] add Xagent link by @yiboyasss in https://github.com/xorbitsai/inference/pull/4693
* FEAT: [UI] remove featured/all toggle and prioritize featured models by @yiboyasss in https://github.com/xorbitsai/inference/pull/4694
* feat(vllm): support v0.18.0 by @llyycchhee in https://github.com/xorbitsai/inference/pull/4718
* FEAT: add gpu load metrics by @leslie2046 in https://github.com/xorbitsai/inference/pull/4712
* feat: Upgrade the base image to version 0.17.1 and add support for aarch64 version images by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4726
* feat(ci): fix aarch64 build by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4735
## Enhancements
* ENH: update model "qwen3.5" JSON by @qinxuye in https://github.com/xorbitsai/inference/pull/4689
* ENH: update model "qwen3.5" JSON by @llyycchhee in https://github.com/xorbitsai/inference/pull/4707
* ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4710
* ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4713
* enh: adapt `normalize` param of vllm>0.16.0 for embedding models. by @la1ty in https://github.com/xorbitsai/inference/pull/4729
* BLD: Requirements dependency version adjustment by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4736
* bld: Requirements dependency version adjustment by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4737
* bld: Requirements dependency version adjustment by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4738
* REF: parallelize supervisor model registration listing by @leslie2046 in https://github.com/xorbitsai/inference/pull/4690
## Bug fixes
* BUG: Fix async client FormData handling and response lifecycle issues by @qinxuye in https://github.com/xorbitsai/inference/pull/4687
* BUG: MLX backend accumulates intermediate generation steps into final output (tested on 1.17.0, 2.0.0, 2.1.0) #4615 by @nasircsms in https://github.com/xorbitsai/inference/pull/4617
* fix(worker): inject parent site-packages into child venv via .pth file by @nasircsms in https://github.com/xorbitsai/inference/pull/4692
* BUG: launch multi gpu qwen3.5 error by @llyycchhee in https://github.com/xorbitsai/inference/pull/4700
* fix(tool_call): add qwen3.5 by @llyycchhee in https://github.com/xorbitsai/inference/pull/4703
* fix(qwen3.5): support tool calls by @llyycchhee in https://github.com/xorbitsai/inference/pull/4709
* FIX: qwen3.5 reasoning parse by @llyycchhee in https://github.com/xorbitsai/inference/pull/4719
* fix(qwen3.5): support XML-like tool call format in non-streaming mode by @amumu96 in https://github.com/xorbitsai/inference/pull/4715
* FIX: webui crash when gpu_utilization  is none by @leslie2046 in https://github.com/xorbitsai/inference/pull/4728
## Documentation
* DOC: add v2.3.0 release notes by @qinxuye in https://github.com/xorbitsai/inference/pull/4688
* DOC: add xagent in readme by @qinxuye in https://github.com/xorbitsai/inference/pull/4699

## New Contributors
* @nasircsms made their first contribution in https://github.com/xorbitsai/inference/pull/4617
* @octo-patch made their first contribution in https://github.com/xorbitsai/inference/pull/4704
* @la1ty made their first contribution in https://github.com/xorbitsai/inference/pull/4729

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v2.3.0...v2.4.0