Starstruk Logo MarkStarstruk

v0.8.23

dancasas/SMPLitexv0.8.23Jun 25, 2026by EricLBuehler

AI Summary

This release focuses on expanding model support (Hunyuan, Gemma4) and enhancing the quantization pipeline with ISQ, alongside dependency updates and bug fixes for the scheduler.

Key Highlights

  • Support for Hunyuan v1 dense and MoE models
  • Optimization for Gemma4 by skipping shared KV projections
  • Addition of ISQ executor and planning
  • CUDA 13.3 support
  • Fix for PagedAttentionScheduler priority logic

New Features

  • CUDA 13.3 support
  • Bump candle dependency
  • Improve CLI docs for --quant and --isq
  • Gemma4 optimization (skip loading projections for shared kv layers)
  • ISQ executor and planning
  • Hunyuan v1 dense and MoE support
  • Fix handling when there are preexisting installs
  • Fix reversed FCFS priority in PagedAttentionScheduler
  • Validate GGUF special token ids against vocab

Full Release Notes

## What's Changed
* feat(cuda): support CUDA 13.3 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2275
* chore: bump candle dep by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2276
* feat(cli): improve --quant and --isq docs in cli by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2277
* feat(gemma4): do not load projections for shared kv layers by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2281
* feat(quant): add isq executor and planning by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2283
* feat: add Hunyuan v1 dense and MoE support by @ASheng1019 in https://github.com/EricLBuehler/mistral.rs/pull/2268
* feat(install): fix handling when there are preexisting installs by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2284
* feat(gdn): add isq support by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2285
* Fix reversed FCFS priority in PagedAttentionScheduler preemption by @pjdurden in https://github.com/EricLBuehler/mistral.rs/pull/2250
* Validate GGUF special token ids against vocab to prevent OOB panic by @pjdurden in https://github.com/EricLBuehler/mistral.rs/pull/2282

## New Contributors
* @ASheng1019 made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/2268
* @pjdurden made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/2250

**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.8.22...v0.8.23