Starstruk Logo MarkStarstruk

v0.2.1

EricLBuehler/mistral.rsv0.2.1Jul 23, 2024by github-actions[bot]

AI Summary

Fixes path normalization, adds Mistral Nemo support, and adds server throughput logging.

Key Highlights

  • Path normalization fix for mistralrs-paged-attn
  • Mistral Nemo support
  • Server throughput logging
  • Optional plotly feature

New Features

  • Mistral Nemo support
  • Server throughput logging
  • Optional plotly

Full Release Notes

## What's Changed
* Fix path normalize for mistralrs-paged-attn by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/592
* ISQ python example by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/593
* Add support for mistral nemo by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/595
* Fix dtype with QLinear by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/600
* Update paged-attn build.rs with NVCC flags by @joshpopelka20 in https://github.com/EricLBuehler/mistral.rs/pull/604
* Bump openssl from 0.10.64 to 0.10.66 by @dependabot in https://github.com/EricLBuehler/mistral.rs/pull/605
* Update GitHub issue templates by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/607
* Add server throughput logging by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/608
* Make the plotly feature optional by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/597
* Use OnceLock for Python bindings device by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/602
* Topk for X-LoRA scalings by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/609
* Fix server cross-origin errors by @openmynet in https://github.com/EricLBuehler/mistral.rs/pull/610
* Refactor sampler by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/611
* Bump version to 0.2.1 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/613

## New Contributors
* @dependabot made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/605
* @openmynet made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/610

**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.2.0...v0.2.1

## Install mistralrs-server 0.2.1

### Install prebuilt binaries via shell script

```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.1/mistralrs-server-installer.sh | sh
```

## Download mistralrs-server 0.2.1

|  File  | Platform | Checksum |
|--------|----------|----------|
| [mistralrs-server-aarch64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.1/mistralrs-server-aarch64-apple-darwin.tar.xz) | Apple Silicon macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.1/mistralrs-server-aarch64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.1/mistralrs-server-x86_64-apple-darwin.tar.xz) | Intel macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.1/mistralrs-server-x86_64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-unknown-linux-gnu.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.1/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz) | x64 Linux | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.1/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz.sha256) |