Starstruk Logo MarkStarstruk

v0.1.17

EricLBuehler/mistral.rsv0.1.17Jun 11, 2024by EricLBuehler

AI Summary

Focuses on GGUF format support (Mixtral experts, chat templates), Phi-3 vision, and documentation.

Key Highlights

  • Support GGUF Mixtral format where experts are in one tensor
  • Exposed Phi3V loader
  • Implemented GPT2 (BPE) GGUF tokenizer conversion
  • Support chat template from GGUF

New Features

  • GGUF Mixtral expert support
  • GPT2 tokenizer conversion
  • GGUF chat templates

Full Release Notes

## What's Changed
* Add and update template READMEs by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/405
* Improve Rust crates docs by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/406
* Expose phi3v loader and remove unused deps by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/408
* Support GGUF Mixtral format where experts are in one tensor by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/355
* Refactor with normal loading metadata for vision models by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/409
* Phi 3 vision ISQ support by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/410
* Remove causal masks cache by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/412
* Fix: use new slice_assign by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/415
* Fix Phi-3 GGUF by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/414
* Implement gpt2 (BPE) GGUF tokenizer conversion by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/397
* Support chat template from GGUF by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/416
* Expose API to specify dtype during loading by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/417
* Lock candle version to commit by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/419
* Bump version to 0.1.17 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/420


**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.1.16...v0.1.17