Starstruk Logo MarkStarstruk

v1.2.2

xorbitsai/inferencev1.2.2Feb 8, 2025by XprobeBot

AI Summary

This release adds support for Qwen2.5-VL-instruct, InternLM3, DeepSeek-R1-Distill-Llama, Kokoro-82M, and updates vLLM support for Qwen2.5-VL-instruct. It also fixes compatibility issues with llama.cpp quantizations and transformers continuous batching.

Key Highlights

  • Support for Qwen2.5-VL-instruct, InternLM3, and DeepSeek-R1-Distill-Llama.
  • Kokoro-82M text-to-speech support.
  • vLLM support for Qwen2.5-VL-instruct.
  • Fixed llama.cpp quantization handling.
  • Updated transformers continuous batching compatibility.

New Features

  • support qwen2.5-vl-instruct
  • Support internlm3
  • support deepseek-r1-distill-llama
  • Support Kokoro-82M
  • vllm support for qwen2.5-vl-instruct

Full Release Notes

# What's new in 1.2.2 (2025-02-08)
These are the changes in inference v1.2.2.
## New features
* FEAT: support qwen2.5-vl-instruct by @qinxuye in https://github.com/xorbitsai/inference/pull/2788
* FEAT: Support internlm3 by @Jun-Howie in https://github.com/xorbitsai/inference/pull/2789
* FEAT: support deepseek-r1-distill-llama by @qinxuye in https://github.com/xorbitsai/inference/pull/2811
* FEAT: Support Kokoro-82M by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2790
* FEAT: vllm support for qwen2.5-vl-instruct by @qinxuye in https://github.com/xorbitsai/inference/pull/2821
## Bug fixes
* BUG: fix llama-cpp when some quantizations have multiple parts by @qinxuye in https://github.com/xorbitsai/inference/pull/2786
* BUG: Use `Cache` class instead of raw `tuple` for transformers continuous batching, compatible with latest `transformers` by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/2820
## Documentation
* DOC: Update multimodal doc by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2785
* DOC: update model docs by @qinxuye in https://github.com/xorbitsai/inference/pull/2792
* DOC: fix docs by @qinxuye in https://github.com/xorbitsai/inference/pull/2793
* DOC: Fix a couple of typos by @Paleski in https://github.com/xorbitsai/inference/pull/2817

## New Contributors
* @Paleski made their first contribution in https://github.com/xorbitsai/inference/pull/2817

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.2.1...v1.2.2