Starstruk Logo MarkStarstruk

v0.12.0

xorbitsai/inferencev0.12.0Jun 7, 2024by XprobeBot

AI Summary

A major feature release introducing several new models including Mini-CPM-Llama3, GLM4, Mistral, Codestral, and Qwen2. It also adds ChatTTS support and continuous batching for chat models on the transformers backend.

Key Highlights

  • New models: Mini-CPM-Llama3, GLM4, Mistral, Codestral, Qwen2
  • ChatTTS support
  • Continuous batching for chat models
  • GLM-4V 9b support

New Features

  • new model: mini-cpm-llama3-v-2.5
  • support glm4-chat & glm4-chat-1m
  • add mistral-instruct-v0.3
  • add codestral-v0.1
  • Support ChatTTS
  • Continuous batching for chat model on transformers backend
  • support qwen2
  • Feat: support glm-4v 9b

Full Release Notes

# What's new in 0.12.0 (2024-06-07)
These are the changes in inference v0.12.0.
## New features
* FEAT: new model: mini-cpm-llama3-v-2.5 by @Minamiyama in https://github.com/xorbitsai/inference/pull/1577
* FEAT: support glm4-chat & glm4-chat-1m by @qinxuye in https://github.com/xorbitsai/inference/pull/1584
* FEAT: add mistral-instruct-v0.3 by @qinxuye in https://github.com/xorbitsai/inference/pull/1576
* FEAT: add codestral-v0.1 by @qinxuye in https://github.com/xorbitsai/inference/pull/1575
* FEAT: Support ChatTTS by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1578
* FEAT: Continuous batching for chat model on transformers backend by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1548
* FEAT: support qwen2 by @qinxuye in https://github.com/xorbitsai/inference/pull/1597
* Feat: support glm-4v 9b by @Minamiyama in https://github.com/xorbitsai/inference/pull/1591
## Enhancements
* ENH: make CogVLM2 support stream output by @Minamiyama in https://github.com/xorbitsai/inference/pull/1572
* BLD: Docker clean all images after building image on self-hosted machine by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1595
* BLD: Fix pip is looking multiple versions of some packages while installing by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1603
## Bug fixes
* BUG: Fix typo for cogvlm2 by @Minamiyama in https://github.com/xorbitsai/inference/pull/1573
## Documentation
* DOC: added new models in README by @qinxuye in https://github.com/xorbitsai/inference/pull/1585
* DOC: Fix audio doc by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1593
* DOC: Usage about cal-model-memory by @wxiwnd in https://github.com/xorbitsai/inference/pull/1589
* DOC: Fix audio doc by @codingl2k1 in https://github.com/xorbitsai/inference/pull/1599
* DOC: Continuous batching by @ChengjieLi28 in https://github.com/xorbitsai/inference/pull/1602
* DOC: add new models to readme by @qinxuye in https://github.com/xorbitsai/inference/pull/1604

## New Contributors
* @wxiwnd made their first contribution in https://github.com/xorbitsai/inference/pull/1589

**Full Changelog**: https://github.com/xorbitsai/inference/compare/v0.11.3...v0.12.0