Starstruk Logo MarkStarstruk

v0.10.2

xorbitsai/inferencev0.10.2Apr 19, 2024by XprobeBot

AI Summary

This release introduces replica configuration for embeddings and rerank models, supports multi-LoRA usage, and adds several new models like SeaLLM-7B and BGE rerankers.

Key Highlights

  • Replica configuration for embeddings and rerank models
  • LoRa multi-support
  • Support for SeaLLM-7B and c4ai-command-r-v01
  • Support for BAAI/bge-reranker-v2
  • UI hotkey for search

New Features

  • [UI] Add replica configuration when launching `embedding` and `rerank` models
  • Lora multi support
  • Support SeaLLM-7B and c4ai-command-r-v01
  • Support BAAI/bge-reranker-v2-* rerank model
  • UI supports multi lora
  • support m3e embedding models
  • hotkey to active search
  • support codeqwen1.5-chat

Full Release Notes

# What's new in 0.10.2 (2024-04-19)
These are the changes in inference v0.10.2.
## New features
* FEAT: [UI] Add replica configuration when launching `embedding` and `rerank` models by @yiboyasss in https://github.com/xorbitsai/inference/pull/1306
* FEAT: Lora multi support by @hainaweiben in https://github.com/xorbitsai/inference/pull/1273
* FEAT: Support SeaLLM-7B and c4ai-command-r-v01 by @mujin2 in https://github.com/xorbitsai/inference/pull/1310
* FEAT: Support BAAI/bge-reranker-v2-* rerank model by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1305
* FEAT: UI supports multi lora by @yiboyasss in https://github.com/xorbitsai/inference/pull/1320
* FEAT: Add_cia4command_modelscope by @mujin2 in https://github.com/xorbitsai/inference/pull/1321
* FEAT: support m3e embedding models by @qinxuye in https://github.com/xorbitsai/inference/pull/1298
* FEAT: hotkey to active search by @Minamiyama in https://github.com/xorbitsai/inference/pull/1287
* FEAT: support codeqwen1.5-chat by @qinxuye in https://github.com/xorbitsai/inference/pull/1322
## Enhancements
* ENH: Support custom audio model by @amumu96 in https://github.com/xorbitsai/inference/pull/1279
* ENH: support int and str compare for model size by @mikeshi80 in https://github.com/xorbitsai/inference/pull/1277
* BLD: Add `FlagEmbedding` in cpu docker by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1318
* REF: support query for engine feature by @Ago327 in https://github.com/xorbitsai/inference/pull/1294
## Others
* Revert "REF: support query for engine feature" by @qinxuye in https://github.com/xorbitsai/inference/pull/1329


**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.10.1...v0.10.2