Starstruk Logo MarkStarstruk

v0.9.3

EricLBuehler/mistral.rsv0.9.3Sep 7, 2026by EricLBuehler

AI Summary

This release focuses on significant performance optimizations through first-class FP8 support, concurrent decoding, and scheduler improvements. It also standardizes server error codes, introduces new metrics, and adds support for NVFP4 and ModelOpt checkpoints.

Key Highlights

  • First-class FP8 support with concurrent DFlash decode and tensor-core gemv/gemm
  • Server error codes aligned to the OpenAI standard
  • Support for NVFP4 checkpoints, ModelOpt, and compressed-tensors
  • New metrics and logging for prefill and decode throughput
  • Improved Anthropic Messages and Claude Code support

New Features

  • FP8 fused MoE and load-time kernel autotuner
  • Separate prefill and decode throughput logging
  • Mistralrs_sequences_capacity gauge
  • NVFP4 checkpoint loading and cuTile Blackwell acceleration
  • ModelOpt and compressed-tensors checkpoint support
  • Anthropic Messages and Claude Code API support
  • GB10 (Blackwell) build support restored

Full Release Notes

## What's Changed
* feat(core): concurrency fixes and scheduler improvements by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2389
* perf: first-class FP8 and concurrent DFlash decode by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2390
* feat(metrics): add mistralrs_sequences_capacity gauge by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2391
* perf(core): optimize exact stochastic DFlash serving by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2392
* feat(server): align server error codes to openai standard by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2393
* feat(logging): expose separate prefill and decode t/s logging by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2395
* fix(multimodal): keep Qwen media work out of decode by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2396
* fix(ci): green the build for rust 1.98 and metal by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2397
* feat(core): report cached prompt tokens in usage by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2398
* perf(core): improve hybrid FP8 serving by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2399
* feat(server): improve Anthropic Messages and Claude Code support by @matt-minev in https://github.com/EricLBuehler/mistral.rs/pull/2400
* docs: refresh generated reference artifacts by @matt-minev in https://github.com/EricLBuehler/mistral.rs/pull/2401
* fix(website): fix handling for clipboard by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2403
* perf(cuda): scale FP8 and DFlash2 serving through C128 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2404
* feat(website): add details about v0.9.3 release by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2405
* chore(cuda): clippy by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2406
* feat(release): improve reliability, add caching by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2408
* fix(core): align tool chat template JSON rendering by @atirna in https://github.com/EricLBuehler/mistral.rs/pull/2371
* fix(lfm2): support legacy full_attn_idxs config field by @tugot17 in https://github.com/EricLBuehler/mistral.rs/pull/2305
* fix(webui): send chat images as data URLs (fix HTTP 500 on image send) by @subin9 in https://github.com/EricLBuehler/mistral.rs/pull/2338
* feat(core): bump cutile to v0.3.0 by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2410
* feat(core): fp8 tensor-core gemv/gemm, cutile gdn prefill by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2412
* feat(cutile): add fp8 fused moe and a load-time kernel autotuner by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2413
* fix(cuda): restore GB10 builds with cudaforge 0.1.6 by @sizzlecar in https://github.com/EricLBuehler/mistral.rs/pull/2415
* feat(fp8): support ModelOpt and compressed-tensors checkpoints with cutile kernels by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2416
* feat(quant): add NVFP4 checkpoint loading and cuTile Blackwell acceleration by @EricLBuehler in https://github.com/EricLBuehler/mistral.rs/pull/2418

## New Contributors
* @atirna made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/2371
* @tugot17 made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/2305
* @subin9 made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/2338
* @sizzlecar made their first contribution in https://github.com/EricLBuehler/mistral.rs/pull/2415

**Full Changelog**: https://github.com/EricLBuehler/mistral.rs/compare/v0.9.2...v0.9.3