Starstruk Logo MarkStarstruk

v1.6.0

xorbitsai/inferencev1.6.0May 16, 2025by XprobeBot

AI Summary

A major release introducing new models (XiYanSQL, HuatuoGPT-o1, DianJin-R1), image-to-video generation, and extensive quantization support (AWQ, GPTQ).

Key Highlights

  • Introduction of XiYanSQL, HuatuoGPT-o1, and DianJin-R1 models
  • Image-to-video generation support
  • Qwen3-235B-A22B GPTQ Quantization (Int4/Int8)
  • New Video and Audio UI interfaces

New Features

  • XiYanSQL-QwenCoder-2504 model
  • HuatuoGPT-o1 model
  • DianJin-R1 model
  • Image-to-video generation support
  • Qwen3-235B-A22B GPTQ Quantization Int4/Int8
  • Video UI interface
  • Audio UI interface
  • Auto add 'm' tag when missed
  • Skywork-OR1 support
  • Skywork-OR1 GPTQ for 32B
  • Enable_thinking parameter for loading qwen3
  • Qwen2.5-Omni-3B support
  • MLX format for qwen3
  • Qwen3-AWQ for 14B & 32B
  • QwQ uses unsloth gguf
  • llama.cpp backend uses xllamacpp
  • UI documentation link button
  • UI version info display
  • Worker environment isolation
  • Isolate autoawq and GPTQModel into separate extra install

Full Release Notes

# What's new in 1.6.0 (2025-05-16)
These are the changes in inference v1.6.0.
## New features
* FEAT: [MODEL]XiYanSQL-QwenCoder-2504 by @Minamiyama in https://github.com/xorbitsai/inference/pull/3352
* FEAT: [Model]HuatuoGPT-o1 by @Minamiyama in https://github.com/xorbitsai/inference/pull/3353
* FEAT: [Model]DianJin-R1 by @Minamiyama in https://github.com/xorbitsai/inference/pull/3343
* FEAT: support image_to_video by @qinxuye in https://github.com/xorbitsai/inference/pull/3386
* FEAT: Qwen3-235B-A22B GPTQ Quantization Int4 Int8 by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3422
* FEAT: use xo.wait_for instead of asyncio.wait_for for actor call by @qinxuye in https://github.com/xorbitsai/inference/pull/3439
* FEAT: video UI by @qinxuye in https://github.com/xorbitsai/inference/pull/3448
* FEAT: auto add <think> tag when it is missed by @amumu96 in https://github.com/xorbitsai/inference/pull/3456
* FEAT: Support Skywork-OR1 by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3447
* FEAT: audio UI by @qinxuye in https://github.com/xorbitsai/inference/pull/3457
* FEAT: Support Skywork-OR1 gptq for 32B by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3464
* FEAT: support enable_thinking for loading qwen3 by @qinxuye in https://github.com/xorbitsai/inference/pull/3463
## Enhancements
* ENH: Qwen/Qwen2.5-Omni-3B by @Minamiyama in https://github.com/xorbitsai/inference/pull/3366
* ENH: added mlx format for qwen3 & update docs by @qinxuye in https://github.com/xorbitsai/inference/pull/3369
* ENH: Qwen3-AWQ for 14B & 32B by @Minamiyama in https://github.com/xorbitsai/inference/pull/3370
* ENH: Update the activated_steze_In_billions parameter in the deepseek-vl2 model by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3380
* ENH: add mlx-community/Qwen2.5-VL-32B-Instruct by @xiaohan815 in https://github.com/xorbitsai/inference/pull/3405
* ENH: [UI] add a documentation link button in side menu by @Minamiyama in https://github.com/xorbitsai/inference/pull/3411
* ENH: QwQ use unsloth gguf by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3408
* ENH: llama.cpp backend use xllamacpp by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3412
* ENH: [UI] display version info in side menu by @Minamiyama in https://github.com/xorbitsai/inference/pull/3423
* ENH: Worker env isolation by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3362
* ENH: Use Qwen's official quantitative model repository by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3436
* ENH: Update cosyvoice by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3365
* BLD: isolate autoawq and GPTQModel into separate extra install by @qinxuye in https://github.com/xorbitsai/inference/pull/3397
* BLD: pin transformers version at 4.51.3 by @amumu96 in https://github.com/xorbitsai/inference/pull/3431
* REF: support loading model config in function by @Minamiyama in https://github.com/xorbitsai/inference/pull/3428
## Bug fixes
* BUG: fix qwen3 235b spec by @qinxuye in https://github.com/xorbitsai/inference/pull/3375
* BUG: fix incomplete parsing of reasoning content in reasoning_parser by @amumu96 in https://github.com/xorbitsai/inference/pull/3391
* BUG: fix the processing logic for inference content parsing and tool calls by @amumu96 in https://github.com/xorbitsai/inference/pull/3394
* BUG: fix stop word handling logic in vllm model generation configuration by @amumu96 in https://github.com/xorbitsai/inference/pull/3414
* BUG: fix Model._get_full_prompt() takes 3 positional arguments but 4 were given by @qinxuye in https://github.com/xorbitsai/inference/pull/3417
* BUG: fix potential stop hang by @qinxuye in https://github.com/xorbitsai/inference/pull/3434
* BUG: [UI] Added cpu_offload parameter to video model and fixed bug in audio model's filtering function. by @yiboyasss in https://github.com/xorbitsai/inference/pull/3461

## New Contributors
* @xiaohan815 made their first contribution in https://github.com/xorbitsai/inference/pull/3405

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.5.1...v1.6.0