v0.1.25

EricLBuehler/mistral.rsv0.1.25Jul 3, 2024by github-actions[bot]

AI Summary

Major feature release introducing AnyMoE (Mixture of Experts), Starcoder 2 model, vision interactive mode, and optimized loading speeds.

Key Highlights

  • Added AnyMoE (Mixture of Experts) support
  • Added Starcoder 2 model
  • Added vision interactive mode
  • Added X-LoRA/LoRA support for Gemma 2
  • Optimized local loading speed

New Features

  • AnyMoE support
  • Starcoder 2 model
  • Vision interactive mode
  • Gemma 2 X-LoRA support

Full Release Notes

## Summary
- Added [AnyMoE](https://github.com/EricLBuehler/mistral.rs/blob/master/docs/ANYMOE.md)
- Added the [Starcoder 2](https://github.com/EricLBuehler/mistral.rs/pull/522) model
- [Vision interactive mode](https://github.com/EricLBuehler/mistral.rs/pull/468)
- [X-LoRA and LoRA support for Gemma 2](https://github.com/EricLBuehler/mistral.rs/pull/499)
- [Optimized local loading speed](https://github.com/EricLBuehler/mistral.rs/pull/523)
- [Release using `cargo dist`](https://github.com/EricLBuehler/mistral.rs/pull/480)

## MSRV
`cargo msrv`

MSRV of mistral.rs is `1.75.0`.

## What's Changed
* Add a vision interactive mode by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/468
* Add support for X-LoRA/LoRA to Gemma 2 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/499
* Add adapter activation support for Gemma 2 LoRA by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/500
* Decrease MSRV to 1.75.0 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/501
* Fix docs with preprocessor config by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/502
* Allow for optional `model` field in OpenAI requests by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/504
* Support streaming for completion by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/508
* Select the best device in examples by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/507
* Handle Metal error from bf16 autodtype selection by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/509
* Use chat messages for the rust examples and show T/s by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/510
* Avoid f64 to f32 cast in phirope for dtype on metal by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/512
* Handle errors in rust examples by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/513
* AnyMoE: Build an MoE model from anything, quickly by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/476
* chore: update paths.rs by @eltociear in https://github.com/EricLBuehler/mistral.rs/pull/516
* Add AnyMoE support for vision models by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/515
* Support images in AnyMoE dataset by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/517
* Handle metal case for nonzero and bitwise ops by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/518
* Support saving and loading AnyMoE gating layer by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/519
* Implement the Starcoder 2 model architecture by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/522
* Docs for Starcoder 2 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/524
* Starcoder2 docs by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/525
* Export loader for starcoder2 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/526
* Support the new phi3 models by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/530
* Source the AnyMoE dataset from a structured JSON file by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/531
* Create an AnyMoE loss graph by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/532
* Add AnyMoE demo video by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/533
* Add judgment for whether model_id is a local path to speed up local model loading by @chenwanqq in https://github.com/EricLBuehler/mistral.rs/pull/523
* Update docs by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/534
* release using cargo-dist by @kranurag7 in https://github.com/EricLBuehler/mistral.rs/pull/480
* Bump version to 0.1.25 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/535

## New Contributors
* @kranurag7 made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/480

**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.1.24...v0.1.25


## Download mistralrs-server 0.1.25

|  File  | Platform | Checksum |
|--------|----------|----------|
| [mistralrs-server-aarch64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.25/mistralrs-server-aarch64-apple-darwin.tar.xz) | Apple Silicon macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.25/mistralrs-server-aarch64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-apple-darwin.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.25/mistralrs-server-x86_64-apple-darwin.tar.xz) | Intel macOS | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.25/mistralrs-server-x86_64-apple-darwin.tar.xz.sha256) |
| [mistralrs-server-x86_64-unknown-linux-gnu.tar.xz](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.25/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz) | x64 Linux | [checksum](https://github.com/EricLBuehler/mistral.rs/releases/download/v0.1.25/mistralrs-server-x86_64-unknown-linux-gnu.tar.xz.sha256) |