Starstruk Logo MarkStarstruk

v1.3.1

xorbitsai/inferencev1.3.1Mar 9, 2025by qinxuye

AI Summary

This release adds support for Qwen2.5-Instruct-1M, Moonlight-16B-A3B, and QwQ-32B models, along with InternVL 2.5. It introduces the `reasoning_content` parameter across all engines and enhances the UI with filtering and copy buttons.

Key Highlights

  • Support for new models: Qwen2.5-Instruct-1M, Moonlight-16B-A3B, QwQ-32B.
  • InternVL 2.5 support.
  • Introduction of `reasoning_content` parameter for all engines.
  • UI enhancements: Add copy button, model ability filtering for audio models.
  • Support for xllamacpp backend.

New Features

  • Support qwen2.5-instruct-1m
  • Support moonlight-16b-a3b
  • create_embedding add field model_replica
  • [UI] add the reasoning_content parameter
  • Support QwQ-32B
  • all engine support reasoning_content

Full Release Notes

# What's new in 1.3.1 (2025-03-09)
These are the changes in inference v1.3.1.
## New features
* FEAT: Support qwen2.5-instruct-1m by @Jun-Howie in https://github.com/xorbitsai/inference/pull/2928
* FEAT: Support moonlight-16b-a3b by @Jun-Howie in https://github.com/xorbitsai/inference/pull/2963
* FEAT: create_embedding add field model_replica by @zhoudelong in https://github.com/xorbitsai/inference/pull/2779
* FEAT: [UI] add the reasoning_content parameter. by @yiboyasss in https://github.com/xorbitsai/inference/pull/2980
* FEAT: Support QwQ-32B by @cyhasuka in https://github.com/xorbitsai/inference/pull/3005
* FEAT: all engine support reasoning_content by @amumu96 in https://github.com/xorbitsai/inference/pull/3013
## Enhancements
* ENH: InternVL2.5-MPO by @Minamiyama in https://github.com/xorbitsai/inference/pull/2913
* ENH: [UI] add copy button by @Minamiyama in https://github.com/xorbitsai/inference/pull/2920
* ENH: [UI] add model ability filtering feature to the audio model. by @yiboyasss in https://github.com/xorbitsai/inference/pull/2986
* ENH: Support xllamacpp by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2997
* BLD: Install ffmpeg 6 for audio & video models by @phuchoang2603 in https://github.com/xorbitsai/inference/pull/2946
* BLD: fix ffprobe library not imported by @phuchoang2603 in https://github.com/xorbitsai/inference/pull/2971
* BLD: fix docker requirements for sglang by @qinxuye in https://github.com/xorbitsai/inference/pull/3015
* REF: [UI] move featureModels to data.js by @yiboyasss in https://github.com/xorbitsai/inference/pull/3008
## Bug fixes
* BUG: fix qwen2.5-vl-7b cannot chat bug by @amumu96 in https://github.com/xorbitsai/inference/pull/2944
* BUG: Fix modelscope model id on Qwen2.5-VL Added support for AWQ quantization format in Qwen2.5-VL by @Jun-Howie in https://github.com/xorbitsai/inference/pull/2943
* BUG: fix Error while using Langchain-chatchat, because the parameter [max_tokens] passed is None by @William533036 in https://github.com/xorbitsai/inference/pull/2962
* BUG: using jina-clip-v2, no attribute error when only text of image pass in by @Minamiyama in https://github.com/xorbitsai/inference/pull/2974
* BUG: fix compatibility of mlx-lm v0.21.5 by @qinxuye in https://github.com/xorbitsai/inference/pull/2993
* BUG: Fix tokenizer error in create_embedding by @shuaiqidezhong in https://github.com/xorbitsai/inference/pull/2992
* BUG: wrong kwargs passing to encode method when using jina-clip-v2 by @Minamiyama in https://github.com/xorbitsai/inference/pull/2991
* BUG: [UI] fix the white screen bug. by @yiboyasss in https://github.com/xorbitsai/inference/pull/3014

## New Contributors
* @phuchoang2603 made their first contribution in https://github.com/xorbitsai/inference/pull/2946
* @William533036 made their first contribution in https://github.com/xorbitsai/inference/pull/2962
* @zhoudelong made their first contribution in https://github.com/xorbitsai/inference/pull/2779

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.3.0.post2...v1.3.1