Starstruk Logo MarkStarstruk

v0.1.26

EricLBuehler/mistral.rsv0.1.26Jul 5, 2024by github-actions[bot]

AI Summary

This release focuses on improvements to device mapping, memory tracking, and the addition of LLaVA support alongside various documentation and installer updates.

Key Highlights

  • Added LLaVA Support
  • Added memory usage tracking and CUDA memory fix
  • Enhanced AnyMoE device mapping and warnings
  • Added benchmark for A6000 GPU

New Features

  • LLaVA Support
  • Memory usage tracking

Full Release Notes

## What's Changed
* Reference cargo dist in readme by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/536
* Fix xlora table in readme by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/537
* Warning about device mapping for anymoe by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/538
* Add updater and installer programs to cargo dist functionality by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/540
* Device mapping for AnyMoE by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/539
* Add tracking of memory usage by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/392
* Fix memory avail for cuda by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/543
* Add LLaVA Support by @chenwanqq in https://github.com/EricLBuehler/mistral.rs/pull/484
* Device mapping fix for normal llama by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/549
* Add benchmark for a6000 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/550
* Bump version to 0.1.26 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/551


**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.1.25...v0.1.26

## Install mistralrs-server 0.1.26

### Install prebuilt binaries via shell script

```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.26/mistralrs-server-installer.sh | sh
```

## Download mistralrs-server 0.1.26

|  File  | Platform | Checksum |
|--------|----------|----------|
| [mistralrs-server-aarch64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.26/mistralrs-server-aarch64-apple-darwin.tar.xz) | Apple Silicon macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.26/mistralrs-server-aarch64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.26/mistralrs-server-x86_64-apple-darwin.tar.xz) | Intel macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.26/mistralrs-server-x86_64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-unknown-linux-gnu.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.26/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz) | x64 Linux | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.26/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz.sha256) |