Starstruk Logo MarkStarstruk

0.1.0

1b5d/llm-api0.1.0Jul 23, 2023by 1b5d

AI Summary

This major release introduces the Huggingface generic model support for running popular HF models, upgrades to run Llama 2, and streamlines Docker images to just two variants. It represents a significant expansion of supported models and improved deployment options.

Key Highlights

  • Introduced Huggingface generic model for running popular HF models
  • Upgraded llama.cpp to support newer Llama models like Llama 2
  • Streamlined docker images to 2 variants: lightweight default and GPU-enabled with CUDA support
  • General fixes and stability improvements

New Features

  • Huggingface generic model support
  • Llama 2 support via llama.cpp upgrade
  • Streamlined Docker image structure

Full Release Notes

- Introducing Huggingface generic model, which can be used to run many popular models on HF
- Upgrade llama.cpp in order to run newer Llama models like Llama 2
- Streamline docker images down to 2 images: a default lightweight one, and a gpu enabled image with nvidia / cuda support
- General fixes and stability improvements