Starstruk Logo MarkStarstruk

v0.15.4

xorbitsai/inferencev0.15.4Oct 12, 2024by XprobeBot

AI Summary

This release focuses on tool calling capabilities for Llama 3.1 and Qwen2.5, adds Whisper-large-v3-turbo audio model, and introduces progress tracking for image models.

Key Highlights

  • Added tool call support for Llama 3.1 and Qwen2.5.
  • Added Whisper-large-v3-turbo audio model.
  • Introduced progress tracking for image model operations.

New Features

  • Llama 3.1 Instruct support tool call
  • qwen2.5 instruct tool call
  • add whisper-large-v3-turbo audio model
  • Add environment variable setting to increase the retry attempts after model download failures
  • support getting progress for image model
  • support qwenvl2 vllm engine

Full Release Notes

# What's new in 0.15.4 (2024-10-12)
These are the changes in inference v0.15.4.
## New features
* FEAT: Llama 3.1 Instruct support tool call by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2388
* FEAT: qwen2.5 instruct tool call by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2393
* FEAT: add whisper-large-v3-turbo audio model by @hwzhuhao in https://github.com/xorbitsai/inference/pull/2409
* FEAT: Add environment variable setting to increase the retry attempts after model download failures by @hwzhuhao in https://github.com/xorbitsai/inference/pull/2411
* FEAT: support getting progress for image model by @qinxuye in https://github.com/xorbitsai/inference/pull/2395
* FEAT: support qwenvl2 vllm engine by @amumu96 in https://github.com/xorbitsai/inference/pull/2428
## Enhancements
* ENH: Launch the ChatTTS model with kwargs by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2425
* REF: refactor controlnet for image model by @qinxuye in https://github.com/xorbitsai/inference/pull/2346
## Bug fixes
* BUG: Pin ChatTTS<0.2 by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2419
* BUG: tool call streaming output has duplicated list by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/2416


**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.15.3...v0.15.4