Starstruk Logo MarkStarstruk

v1.5.0

xorbitsai/inferencev1.5.0Apr 19, 2025by XprobeBot

AI Summary

Major release introducing InternVL3 and Megatts3 models, AWQ quantization, and UI improvements for progress tracking.

Key Highlights

  • Support for InternVL3 and Megatts3 models
  • AWQ quantization support for InternVL3
  • UI progress bar and model launch cancellation
  • Thinking process display in Gradio chat interface

New Features

  • Megatts3 model support
  • InternVL3 model support
  • Paraformer-zh model support
  • SeaLLMs-v3 support
  • Download progress and cancellation support
  • Thinking process display in Gradio chat
  • GLM4-0414 support
  • Min/max_pixels parameters for vision models
  • Skywork-OR1-preview support
  • UI progress bar and functionality to cancel model launch
  • AWQ quantization support for InternVL3
  • Virtualenv support for models
  • Qwen2.5-omni support
  • Compatibility with latest xllamacpp
  • Xllamacpp as default
  • Updated Gradio interface for chat models
  • Gradio default concurrency set to CPU count
  • Transformers model registration simplification

Full Release Notes

# What's new in 1.5.0 (2025-04-19)
These are the changes in inference v1.5.0.
## New features
* FEAT: Support megatts3 by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3224
* FEAT: InternVL3 by @Minamiyama in https://github.com/xorbitsai/inference/pull/3235
* FEAT: support paraformer-zh by @qinxuye in https://github.com/xorbitsai/inference/pull/3236
* FEAT:support SeaLLMs-v3 by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3248
* FEAT: support getting download progress and cancel download by @qinxuye in https://github.com/xorbitsai/inference/pull/3233
* FEAT: add thinking process in gradio chat interface by @amumu96 in https://github.com/xorbitsai/inference/pull/3245
* FEAT: support glm4-0414 by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3251
* FEAT: support min/max_pixels params for vision model by @amumu96 in https://github.com/xorbitsai/inference/pull/3242
* FEAT: support skywork-or1-preview by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3274
* FEAT: [UI] progress bar and functionality to cancel model launch. by @yiboyasss in https://github.com/xorbitsai/inference/pull/3276
* FEAT:Add AWQ quantization support for InternVL3 by @Jun-Howie in https://github.com/xorbitsai/inference/pull/3285
* FEAT: Support virtualenv for models by @qinxuye in https://github.com/xorbitsai/inference/pull/3241
* FEAT: support qwen2.5-omni by @qinxuye in https://github.com/xorbitsai/inference/pull/3279
## Enhancements
* ENH: Compatible with latest xllamacpp by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3181
* ENH: Use xllamacpp by default by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3198
* ENH: update gradio interface for chat model by @amumu96 in https://github.com/xorbitsai/inference/pull/3265
* ENH: Set gradio default concurrency to cpu count by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3278
* BLD: fix compatibility for mlx-lm>=0.22.3 by @qinxuye in https://github.com/xorbitsai/inference/pull/3195
* BLD: upgrade gradio version for docker by @Minamiyama in https://github.com/xorbitsai/inference/pull/3197
* BLD: fix docker build by @qinxuye in https://github.com/xorbitsai/inference/pull/3207
* BLD: remove setuptools limitation in project.toml by @qinxuye in https://github.com/xorbitsai/inference/pull/3212
* BLD: fix docker build by @amumu96 in https://github.com/xorbitsai/inference/pull/3289
* REF: simplify transformers model registration with decorators. by @Minamiyama in https://github.com/xorbitsai/inference/pull/3191
## Bug fixes
* BUG: fix stop hang for vllm engine by @qinxuye in https://github.com/xorbitsai/inference/pull/3202
* BUG: Fix qwq gguf model path by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3232
* BUG: Fix llama cpp backend load model with multiple parts by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3261
## Documentation
* DOC: Add usage doc for kokoro by @codingl2k1 in https://github.com/xorbitsai/inference/pull/3192
* DOC: add doc about virtual env & update models in README by @qinxuye in https://github.com/xorbitsai/inference/pull/3287


**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.4.1...v1.5.0