Starstruk Logo MarkStarstruk

v0.8.23

stanfordnlp/dspyv0.8.23Jun 25, 2026by EricLBuehler

AI Summary

This release introduces support for CUDA 13.3 and Hunyuan v1 models, along with significant improvements to quantization via ISQ (Iterative Quantization). It also includes various bug fixes for scheduler logic and GGUF token validation.

Key Highlights

  • Added support for CUDA 13.3
  • Introduced ISQ executor and planning for quantization
  • Added Hunyuan v1 dense and MoE model support
  • Optimized Gemma 4 by not loading projections for shared KV layers
  • Fixed reversed FCFS priority in PagedAttentionScheduler

New Features

  • CUDA 13.3 support
  • ISQ executor and planning
  • Hunyuan v1 dense and MoE support
  • Gemma 4 shared KV layer optimization
  • GGUF special token id validation

Full Release Notes

## What's Changed
* feat(cuda): support CUDA 13.3 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2275
* chore: bump candle dep by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2276
* feat(cli): improve --quant and --isq docs in cli by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2277
* feat(gemma4): do not load projections for shared kv layers by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2281
* feat(quant): add isq executor and planning by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2283
* feat: add Hunyuan v1 dense and MoE support by @ASheng1019 in https://github.com/EricLBuehler/mistral.rs/pull/2268
* feat(install): fix handling when there are preexisting installs by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2284
* feat(gdn): add isq support by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2285
* Fix reversed FCFS priority in PagedAttentionScheduler preemption by @pjdurden in https://github.com/EricLBuehler/mistral.rs/pull/2250
* Validate GGUF special token ids against vocab to prevent OOB panic by @pjdurden in https://github.com/EricLBuehler/mistral.rs/pull/2282

## New Contributors
* @ASheng1019 made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/2268
* @pjdurden made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/2250

**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.8.22...v0.8.23