b9757

ggml-org/llama.cppb9757Jun 22, 2026by github-actions[bot]same day as b9756

What changed since b9756

low risk

b9757 drops the unconditional softmax and sort from the top-n-sigma sampler, a performance-only change over b9756.

Improvements

  • Top-n-sigma sampling no longer performs an unconditional softmax and sort, cutting per-token sampling overhead.
  • Sampler logic was reworked so the expensive normalization step only happens where actually required.

Upgrade notes

No configuration or API changes are required; this is a drop-in replacement for b9756. Users relying on top-n-sigma sampling should spot-check output quality, since removing the unconditional softmax/sort can subtly shift token selection probabilities. The earlier edit_file append crash fix from b9756 remains in place.

Analyzed Nov 1, 2026

Upgrading?

Release notes summary

Removed the unconditional application of softmax and sorting in the top-n-sigma sampling algorithm.

Key Highlights

  • Remove unconditional softmax+sort
  • Update top-n-sigma sampler logic
  • Performance improvement

Release Notes (excerpt)

<details open> sampling : remove unconditional softmax+sort in top-n-sigma sampler (#22645) </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) **Android:** - Android arm64 (CPU)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories