Starstruk Logo MarkStarstruk

v2.0.0

xorbitsai/inferencev2.0.0Jan 31, 2026by XprobeBot

AI Summary

Major version 2.0.0 release introduces Qwen3-VL family models, MinerU 2.5 OCR model, GLM-4.6, and Z-Image. Adds video GGUF cache support, chat_template.jinja support, and UI improvements with backend API data-driven approach.

Key Highlights

  • Qwen3-VL-Embedding and Reranker models (2B and 8B variants)
  • MinerU 2.5 OCR model introduction
  • GLM-4.6 and Z-Image model support
  • Video GGUF cache_manager.py for caching
  • chat_template.jinja support for prompt templates
  • UI improvements with browser locale as default language
  • Upgraded vLLM base image to version 0.13.0

New Features

  • Qwen3-VL-Embedding-2B
  • Qwen3-VL-Embedding-8B
  • Qwen3-VL-Reranker-2B
  • Qwen3-VL-Reranker-8B
  • MinerU 2.5 OCR
  • GLM-4.6
  • Z-Image
  • Video GGUF cache

Full Release Notes

# What's new in 2.0.0 (2026-01-31)
These are the changes in inference v2.0.0.
## New features
* FEAT: add video gguf cache_manager.py by @OliverBryant in https://github.com/xorbitsai/inference/pull/4462
* FEAT: [model] Qwen3-VL-Embedding-2B support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4469
* FEAT: [UI] move featured to backend API data-driven; remove frontend hardcoding. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4466
* FEAT: [model] Qwen3-VL-Reranker-8B support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4472
* FEAT: llm cache config in model json to skip unnecessary downloads by @OliverBryant in https://github.com/xorbitsai/inference/pull/4480
* FEAT: [UI] add official website and model hub links. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4493
* FEAT: add custom llm models config json analysis by @OliverBryant in https://github.com/xorbitsai/inference/pull/4478
* FEAT: [model] MinerU2.5-2509-1.2B support by @leslie2046 in https://github.com/xorbitsai/inference/pull/4510
* FEAT: Introduce MinerU 2.5 OCR model. by @leslie2046 in https://github.com/xorbitsai/inference/pull/4511
* FEAT: add chat_template.jinja support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4526
* FEAT: support engines for virtualenv by @OliverBryant in https://github.com/xorbitsai/inference/pull/4497
* FEAT: [model] Z-Image support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4546
* FEAT: [model] GLM-4.6 support by @Jun-Howie in https://github.com/xorbitsai/inference/pull/4525
* FEAT: [model] Qwen3-VL-Embedding-8B support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4470
* FEAT: [UI] use browser locale as default language. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4539
* FEAT: [model] Qwen3-VL-Reranker-2B support by @OliverBryant in https://github.com/xorbitsai/inference/pull/4471
## Enhancements
* ENH: update 3 models JSON ("HunyuanVideo", "gme-Qwen2-VL-7B-Instruct", "gme-Qwen2-VL-2B-Instruct") by @OliverBryant in https://github.com/xorbitsai/inference/pull/4464
* ENH: update models JSON [embedding, image, llm, video] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4463
* ENH: update models JSON [llm] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4490
* ENH: update model "Fun-ASR-Nano-2512" JSON by @leslie2046 in https://github.com/xorbitsai/inference/pull/4496
* ENH: update model "Fun-ASR-MLT-Nano-2512" JSON by @leslie2046 in https://github.com/xorbitsai/inference/pull/4498
* ENH: update model "Qwen3-VL-Embedding-2B" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4503
* ENH: update models JSON [embedding, image, llm, rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4524
* ENH: update models JSON [embedding, image, llm, rerank] by @XprobeBot in https://github.com/xorbitsai/inference/pull/4534
* ENH: update model "Qwen3-VL-Embedding-2B" JSON by @OliverBryant in https://github.com/xorbitsai/inference/pull/4552
* BLD: remove Dockerfile for version CU12.4 by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4487
* REF: [UI] remove featureModels array. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4488
## Bug fixes
* BUG: fix has_musa_device error by @OliverBryant in https://github.com/xorbitsai/inference/pull/4477
* BUG:  [xavier] fix xavier hash function to ensure prefix cache hit by @llyycchhee in https://github.com/xorbitsai/inference/pull/4482
* BUG: image/audio/video download hub exclude modelscope by @OliverBryant in https://github.com/xorbitsai/inference/pull/4483
* BUG: [UI] historical parameter backfill bug. by @yiboyasss in https://github.com/xorbitsai/inference/pull/4479
* BUG: deepseek ocr markdown bug by @OliverBryant in https://github.com/xorbitsai/inference/pull/4491
* BUG: new vllm version cannot launch embedding models by @OliverBryant in https://github.com/xorbitsai/inference/pull/4489
* BUG: Failed to download model 'Fun-ASR-MLT-Nano-2512' after multiple retries by @leslie2046 in https://github.com/xorbitsai/inference/pull/4537
* BUG: transformers version < 5.0.0 by @OliverBryant in https://github.com/xorbitsai/inference/pull/4553
* BUG: cachemanager makedirs only init once to prevent from stuck when downloading by @llyycchhee in https://github.com/xorbitsai/inference/pull/4551
## Documentation
* DOC: add v1.17.0 release note by @qinxuye in https://github.com/xorbitsai/inference/pull/4467
* DOC: add limitations for Xavier by @ZhikaiGuo960110 in https://github.com/xorbitsai/inference/pull/4486
* DOC: add v2.0 doc by @OliverBryant in https://github.com/xorbitsai/inference/pull/4545
* DOC: add cudnn/nccl/cusparselt error solution in virtualenv's doc by @OliverBryant in https://github.com/xorbitsai/inference/pull/4556
## Others
* feat：Upgrade the vllm base image to version 0.13.0 by @zwt-1234 in https://github.com/xorbitsai/inference/pull/4522
* CHORE: modify copyright by @OliverBryant in https://github.com/xorbitsai/inference/pull/4494


**Full Changelog**: https://github.com/xorbitsai/inference/compare/v1.17.0...v2.0.0