Starstruk Logo MarkStarstruk

v0.9.2

EricLBuehler/mistral.rsv0.9.2Aug 20, 2026by EricLBuehler

AI Summary

This release focuses on advanced inference capabilities for Qwen models, including MTP support, speculative decoding enhancements (DFlash), and CUDA graph optimizations. It also improves OpenAI server compatibility and adds detailed engine metrics.

Key Highlights

  • Advanced Qwen3.5 features: MTP support and improved video processor
  • Performance optimizations: CUDA graph batch buckets and batched drafter forward
  • Speculative decoding support for DFlash and DFlash 2
  • Improved OpenAI server compatibility
  • Detailed engine metrics for scheduler status

New Features

  • Qwen3.5 MTP support
  • Qwen3.5 improved video processor
  • OpenAI server compatibility for /v1/responses
  • Engine metrics for scheduler status
  • CUDA graph support for MTP
  • In-place pooled recurrent state kernels (GDN)
  • Speculative decoding support for DFlash and DFlash 2

Full Release Notes

## What's Changed
* fix(fav3): misc fixes relating to FAv3 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2369
* feat(qwen3.5): implement better video processor by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2368
* feat(server): improve openai server compatability with /v1/responses by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2370
* feat(qwen3.5): add mtp support by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2372
* fix(qwen3.5): better support for hybrid models and auto device map by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2373
* feat(metrics): add engine metrics for scheduler status by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2374
* fix(qwen3.8): fixes around gdn rollback for qwen3.8 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2377
* feat(mtp): support cuda graphs by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2379
* feat(gdn): in-place pooled recurrent state kernels, fused MLP, f32 gate params by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2382
* feat(cuda): CUDA graph batch buckets by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2384
* feat(metrics): improve engine metrics for ttft/itl and speculative by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2383
* perf(mtp): quantized drafter, device verify, stash-based GDN rollback, size-aware n_predict by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2385
* feat(speculative): DFlash and DFlash 2 block-diffusion draft models by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2386
* perf(dflash): batched drafter forward and adaptive draft depth by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2388


**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.9.1...v0.9.2