b10241

ggml-org/llama.cppb10241Aug 3, 2026by github-actions[bot]same day as b10240

What changed since b10240

low risk

Fixed critical CUDA data races and added double-buffering for performance.

New features

  • Added double-buffering for single-row softmax operations.
  • Added double-buffering for norm operations.

Improvements

  • Optimized CUDA performance by adding memory barriers only in multi-warp scenarios.
  • Improved CUDA performance via double-buffering implementation.

Bug fixes

  • Fixed critical data races in CUDA block_reduce when reusing Shared Memory.

Upgrade notes

This release fixes critical data races in CUDA code. No configuration changes are required, but ensure your CUDA drivers are up to date to benefit from the performance optimizations.

Analyzed Oct 16, 2026

Upgrading?

Release notes summary

Fixed critical data races in CUDA block reduction operations by adding memory barriers and implemented double-buffering for specific operations to improve performance.

Key Highlights

  • Fixed data-races in CUDA block_reduce when reusing Shared Memory (SMEM)
  • Added double-buffering for single-row softmax
  • Added double-buffering for norm operations
  • Added memory barriers only in multi-warp scenarios to optimize performance

Release Notes (excerpt)

<details open> CUDA: Fix data-races when reusing SMEM in block_reduce (#26385) * CUDA: Fix data-races when reusing block_reduce block_reduce currently doesn't resync after reading from SMEM, causing potential data-races when reusing SMEM for multiple reductions. One may consider simply always adding this in block_reduce, but this comes at a potential perf cost * double-buffering for single-row softmax * double-buffering for norm as well * Add comment…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories