v0.2.3

EricLBuehler/mistral.rsv0.2.3Jul 28, 2024by github-actions[bot]

AI Summary

Implements min-p sampling and improves PagedAttention KV cache allocation.

Key Highlights

Min-p sampling implementation
Improved PagedAttention KV cache allocation from context size
Improved penalty context window calculation

New Features

Min-p sampling
PagedAttention improvements

Full Release Notes

## What's Changed
* Implement min-p sampling by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/625
* Tweak handling when PA cannot allocate by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/632
* Update deps by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/633
* Improve penalty context window calculation by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/636
* Allow setting PagedAttention KV cache allocation from context size by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/640
* Bump version to 0.2.3 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/638


**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.2.2...v0.2.3

## Install mistralrs-server 0.2.3

### Install prebuilt binaries via shell script

```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-installer.sh | sh
```

## Download mistralrs-server 0.2.3

|  File  | Platform | Checksum |
|--------|----------|----------|
| [mistralrs-server-aarch64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-aarch64-apple-darwin.tar.xz) | Apple Silicon macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-aarch64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-x86_64-apple-darwin.tar.xz) | Intel macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-x86_64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-unknown-linux-gnu.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz) | x64 Linux | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.3/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz.sha256) |