Starstruk Logo MarkStarstruk

v0.16.1

xorbitsai/inferencev0.16.1Oct 25, 2024by XprobeBot

AI Summary

This release adds support for GPTQ format of Qwen2.5-Coder-7B-Instruct, GOT-OCR2_0, and MLX support for Flux.1, alongside enhancements to ChatTTS and concurrent request handling.

Key Highlights

  • Added support for Qwen2.5-Coder-7B-Instruct in GPTQ format.
  • New OCR model support: GOT-OCR2_0.
  • Added MLX support for Flux.1 image generation.

New Features

  • Add support for Qwen/Qwen2.5-Coder-7B-Instruct gptq format
  • Support GOT-OCR2_0
  • [UI] Image model with the lora_config
  • added MLX support for Flux.1

Full Release Notes

# What's new in 0.16.1 (2024-10-25)
These are the changes in inference v0.16.1.
## New features
* FEAT: Add support for Qwen/Qwen2.5-Coder-7B-Instruct gptq format by @frostyplanet in https://github.com/xorbitsai/inference/pull/2408
* FEAT: Support GOT-OCR2_0 by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2458
* FEAT: [UI] Image model with the lora_config. by @yiboyasss in https://github.com/xorbitsai/inference/pull/2482
* FEAT: added MLX support for Flux.1 by @qinxuye in https://github.com/xorbitsai/inference/pull/2459
## Enhancements
* ENH: Support ChatTTS 0.2 by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2449
* ENH: Pending queue for concurrent requests by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2473
## Bug fixes
* BUG: Remove duplicated call of model_install by @frostyplanet in https://github.com/xorbitsai/inference/pull/2457
* BUG: fix embedding model gte-Qwen2 dimensions by @JinCheng666 in https://github.com/xorbitsai/inference/pull/2479
## Documentation
* DOC: update enterprise doc links by @qinxuye in https://github.com/xorbitsai/inference/pull/2461

## New Contributors
* @JinCheng666 made their first contribution in https://github.com/xorbitsai/inference/pull/2479

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.16.0...v0.16.1