Starstruk Logo MarkStarstruk

v0.11.2

xorbitsai/inferencev0.11.2May 24, 2024by XprobeBot

AI Summary

This release adds support for new models like DeepSeek LLM and CodeQwen1.5, along with a new memory calculation command. It also enhances functionality for querying cached models and updates compatibility with various libraries.

Key Highlights

  • Added DeepSeek LLM and CodeQwen1.5 support
  • Introduced `cal-model-mem` command for memory calculation
  • Added query functionality for cached models
  • Updated compatibility with `huggingface-hub`, `langchain`, and `torch`

New Features

  • Add command cal-model-mem
  • Add deepseek llm and coder base
  • Add codeqwen1.5
  • Auto detect rerank type for unknown rerank type
  • Provide the functionality to query information on various cached models

Full Release Notes

# What's new in 0.11.2 (2024-05-24)
These are the changes in inference v0.11.2.
## New features
* FEAT: Add command cal-model-mem by @frostyplanet in https://github.com/xorbitsai/inference/pull/1460
* FEAT: add deepseek llm and coder base by @qinxuye in https://github.com/xorbitsai/inference/pull/1533
* FEAT: add codeqwen1.5 by @qinxuye in https://github.com/xorbitsai/inference/pull/1535
* FEAT: Auto detect rerank type for unknown rerank type by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1538
* FEAT: Provide the functionality to query information on various cached models hosted on the query node. by @hainaweiben in https://github.com/xorbitsai/inference/pull/1522
## Enhancements
* ENH: Compatible with `huggingface-hub` `v0.23.0` by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1514
* ENH: convert command-r to chat by @qinxuye in https://github.com/xorbitsai/inference/pull/1537
* ENH: Support Intern-VL-Chat model by @amumu96 in https://github.com/xorbitsai/inference/pull/1536
* BLD: adapt to langchain 0.2.x, which has breaking changes by @mikeshi80 in https://github.com/xorbitsai/inference/pull/1521
* BLD: Fix pre commit by @frostyplanet in https://github.com/xorbitsai/inference/pull/1527
* BLD: compatible with torch 2.3.0 by @qinxuye in https://github.com/xorbitsai/inference/pull/1534
## Bug fixes
* BUG: Fix start worker failed due to None device name by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1539
* BUG: Fix gpu_idx allocate error when set replica > 1  by @amumu96 in https://github.com/xorbitsai/inference/pull/1528
## Others
* CHORE: Basic benchmark/benchmark_rerank.py by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1479


**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.11.1...v0.11.2