Starstruk Logo MarkStarstruk

v0.11.1

xorbitsai/inferencev0.11.1May 17, 2024by XprobeBot

AI Summary

This release focuses on stability by adding support for the Yi-1.5 series and refining LoRa adaptation methods. It includes fixes for Docker images, `llama.cpp` models, and various UI enhancements.

Key Highlights

  • Support for Yi-1.5 series models
  • UI enhancements for GPU/CPU specification
  • Refactoring of LoRa adaptation for LLM models
  • Fixes for Docker image crashes and LoRA issues

New Features

  • Support Yi-1.5 series
  • [UI] embedding and rerank support the specified GPU and CPU
  • Refactoring the LoRa adaptation method for the LLM model
  • Add stream_options support

Full Release Notes

# What's new in 0.11.1 (2024-05-17)
These are the changes in inference v0.11.1.
## New features
* FEAT: support Yi-1.5 series by @qinxuye in https://github.com/xorbitsai/inference/pull/1489
* FEAT: [UI] embedding and rerank support the specified GPU and CPU. by @yiboyasss in https://github.com/xorbitsai/inference/pull/1491
## Enhancements
* ENH: Refactoring the LoRa adaptation method for the LLM model. by @hainaweiben in https://github.com/xorbitsai/inference/pull/1470
* ENH: Add stream_options support by @amumu96 in https://github.com/xorbitsai/inference/pull/1508
## Bug fixes
* BUG: fix top_k for vllm backend by @sixsun10 in https://github.com/xorbitsai/inference/pull/1461
* BUG: Docker image issue due to `torchvision` by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1485
* BUG: Docker image crash during startup due to `llama-cpp-python` by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1507
* BUG: Fix prompt is needed when docker image builds by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1512
* BUG: `llama.cpp` model failed when chat due to `lora` by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1513
## Documentation
* DOC: update quick start ipynb by @qinxuye in https://github.com/xorbitsai/inference/pull/1482
* DOC: Update readme for being integrated by RAGFlow by @JinHai-CN in https://github.com/xorbitsai/inference/pull/1493
* DOC: Lora usage by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1506

## New Contributors
* @sixsun10 made their first contribution in https://github.com/xorbitsai/inference/pull/1461
* @JinHai-CN made their first contribution in https://github.com/xorbitsai/inference/pull/1493

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.11.0...v0.11.1