Starstruk Logo MarkStarstruk

v1.2.0

xorbitsai/inferencev1.2.0Jan 10, 2025by XprobeBot

AI Summary

This release adds HunyuanVideo and Hunyuan-DiT text-to-image support, Cline support for vLLM, and UI theme switching/i18n support. It also supports Qwen2VL on Ascend NPU and introduces KV cache sharing between VLLM replicas.

Key Highlights

  • Support for HunyuanVideo and Hunyuan-DiT text-to-image.
  • Cline support for vLLM engine.
  • UI theme switch and i18n support.
  • Qwen2VL support on Ascend NPU.
  • KV cache sharing between VLLM replicas.

New Features

  • support HunyuanVideo
  • support hunyuan-dit text2image
  • support cline for vllm engine
  • [UI] theme switch
  • support qwen2vl run on ascend npu
  • [UI] Add language toggle for i18n support
  • Support cogagent-9b
  • Xavier: Share KV cache between VLLM replicas
  • [UI] Add gguf_quantization, gguf_model_path, and cpu_offload for image models
  • Support Marco-o1

Full Release Notes

# What's new in 1.2.0 (2025-01-10)
These are the changes in inference v1.2.0.
## New features
* FEAT: support HunyuanVideo by @qinxuye in https://github.com/xorbitsai/inference/pull/2721
* FEAT: support hunyuan-dit text2image by @qinxuye in https://github.com/xorbitsai/inference/pull/2727
* FEAT: support cline for vllm engine by @hwzhuhao in https://github.com/xorbitsai/inference/pull/2734
* FEAT: [UI] theme switch by @Minamiyama in https://github.com/xorbitsai/inference/pull/1335
* FEAT: support qwen2vl run on ascend npu by @Xu-pixel in https://github.com/xorbitsai/inference/pull/2741
* FEAT: [UI] Add language toggle for i18n support. by @yiboyasss in https://github.com/xorbitsai/inference/pull/2744
* FEAT: Support cogagent-9b by @amumu96 in https://github.com/xorbitsai/inference/pull/2740
* FEAT: Xavier: Share KV cache between VLLM replicas by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/2732
* FEAT: [UI] Add gguf_quantization, gguf_model_path, and cpu_offload for image models. by @yiboyasss in https://github.com/xorbitsai/inference/pull/2753
* FEAT: Support Marco-o1 by @Jun-Howie in https://github.com/xorbitsai/inference/pull/2749
## Enhancements
* ENH: [UI] Update Button Style and Interaction Logic for Editing Cache in Model Card. by @yiboyasss in https://github.com/xorbitsai/inference/pull/2746
* ENH: Improve error message by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2738
## Bug fixes
* BUG: adapt mlx-vlm v0.1.7 by @qinxuye in https://github.com/xorbitsai/inference/pull/2724
* BUG: pin mlx<0.22.0 to prevent qwen2_vl failing in mlx-vlm by @qinxuye in https://github.com/xorbitsai/inference/pull/2752
## Others
* FIX: [UI] Resolve bug preventing '/' input in model_path. by @yiboyasss in https://github.com/xorbitsai/inference/pull/2747
* FIX: [UI] Fix dark mode background bug. by @yiboyasss in https://github.com/xorbitsai/inference/pull/2748
* CHORE: Update new models in readme by @codingl2k1 in https://github.com/xorbitsai/inference/pull/2713

## New Contributors
* @Xu-pixel made their first contribution in https://github.com/xorbitsai/inference/pull/2741

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.1.1...v1.2.0