Starstruk Logo MarkStarstruk

v1.5.1

xorbitsai/inferencev1.5.1Apr 30, 2025by XprobeBot

AI Summary

Feature release adding Wan 2.1 text-to-video, Qwen3 support, and GGUF v2 support for vLLM.

Key Highlights

  • Wan 2.1 text-to-video generation
  • Full support for Qwen3 and Qwen3MOE
  • GGUF v2 support for vLLM engine

New Features

  • Wan 2.1 text2video model
  • UI highlight for input box content
  • UI display of model_ability parameter
  • GGUFv2 support for vLLM
  • Ovis2 model
  • Qwen3 and Qwen3MOE support
  • Qwen3 GPTQ quantization format support
  • SSE ping attempts configuration
  • GLM4-0414 MLX and GGUF support
  • Optimized qwen3 with chat_template_kwargs for all engines
  • Unified audio model abilities

Full Release Notes

# What's new in 1.5.1 (2025-04-30)
These are the changes in inference v1.5.1.
## New features
* FEAT: Wan 2.1 text2video by @qinxuye in https://github.com/xorbitsai/inference/pull/3297
* FEAT: [UI] highlight the input box content. by @yiboyasss in https://github.com/xorbitsai/inference/pull/3306
* FEAT: [UI] display the model_ability parameter. by @yiboyasss in https://github.com/xorbitsai/inference/pull/3308
* FEAT: add ggufv2 support for vLLM by @harryzwh in https://github.com/xorbitsai/inference/pull/3259
* FEAT: ovis2 by @Minamiyama in https://github.com/xorbitsai/inference/pull/3170
* FEAT: support Qwen3 and Qwen3MOE by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3347
* FEAT: Add support for Qwen3 GPTQ quantization format by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3363
## Enhancements
* ENH: support setting sse ping attempts by @llyycchhee in https://github.com/xorbitsai/inference/pull/3313
* ENH: Support GLM4-0414 MLX and GGUF by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3325
* ENH: optimize qwen3, support chat_template_kwargs for all engines by @qinxuye in https://github.com/xorbitsai/inference/pull/3354
* REF: Drop internal compression logic for `transformers` quantization, using bnb config instead by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/3324
* REF: Unify audio model abilities by @llyycchhee in https://github.com/xorbitsai/inference/pull/3351
## Bug fixes
* BUG: fix sglang chat by @qinxuye in https://github.com/xorbitsai/inference/pull/3326
* BUG: Show engine options on UI even if the specific engine is not installed by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/3331
* BUG: fix failure of clearing resources when loading model failed by @qinxuye in https://github.com/xorbitsai/inference/pull/3361
## Documentation
* DOC: update troubleshooting.rst for the launch error caused by numpy by @qiulang in https://github.com/xorbitsai/inference/pull/3342

## New Contributors
* @llyycchhee made their first contribution in https://github.com/xorbitsai/inference/pull/3313
* @harryzwh made their first contribution in https://github.com/xorbitsai/inference/pull/3259
* @qiulang made their first contribution in https://github.com/xorbitsai/inference/pull/3342

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.5.0...v1.5.1