Starstruk Logo MarkStarstruk

v0.2.4

EricLBuehler/mistral.rsv0.2.4Aug 1, 2024by github-actions[bot]

AI Summary

Fixes Metal build issues and implements tool calling functionality.

Key Highlights

  • Metal build fixes
  • Tool calling implementation
  • Improved throughput for interactive mode
  • Mistral Nemo support in GGUF

New Features

  • Tool calling
  • Mistral Nemo support
  • Interactive mode throughput

Full Release Notes

## What's Changed
* fix build on metal by returning Device by @rgbkrk in https://github.com/EricLBuehler/mistral.rs/pull/642
* Add invite to Matrix chatroom by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/644
* Make sure we don't have dead links by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/647
* Fix more links by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/648
* Throughput for interactive mode by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/655
* Implement tool calling by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/649
* Fix device map check for paged attn by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/656
* Fix for mistral nemo in gguf by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/657
* Fix check of cache config when device mapping + PA by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/658
* Biollama in tool calling example by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/659
* Biollama in tool calling example by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/660
* Examples for simple tool calling by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/661
* Bump version to 0.2.4 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/662

## New Contributors
* @rgbkrk made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/642

**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.2.3...v0.2.4

## MSRV

MSRV is 1.75

## Install mistralrs-server 0.2.4

### Install prebuilt binaries via shell script

```sh
curl --proto '=https' --tlsv1.2 -LsSf https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-installer.sh | sh
```

## Download mistralrs-server 0.2.4

|  File  | Platform | Checksum |
|--------|----------|----------|
| [mistralrs-server-aarch64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-aarch64-apple-darwin.tar.xz) | Apple Silicon macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-aarch64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-x86_64-apple-darwin.tar.xz) | Intel macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-x86_64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-unknown-linux-gnu.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz) | x64 Linux | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.2.4/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz.sha256) |