Starstruk Logo MarkStarstruk

0.1.2

1b5d/llm-api0.1.2Nov 13, 2023by 1b5d

AI Summary

This release adds support for the AutoAWQ model quantization format and significantly improves Docker deployment flexibility with multiple BLAS backend options. It also cleans up the configuration approach by removing the default config.yaml and providing an example file instead.

Key Highlights

  • Added autoawq model family support
  • Published multiple docker images supporting multiple BLAS backends (llama.cpp)
  • Removed config.yaml and provided config.yaml.example

New Features

  • AutoAWQ model family support
  • Multiple BLAS backend docker images

Full Release Notes

- Added `autoawq` model family
- Published multiple docker images supporting multiple BLAS backends (llama.cpp)
- Remove `config.yaml` and provide `config.yaml.example`