b9948
ggml-org/llama.cppb9948Jul 10, 2026by github-actions[bot]same day as b9947
What changed since b9947
CUDA top-k and argsort now process data in chunks, cutting temporary buffer memory use.
Improvements
- CUDA ggml_top_k() and ggml_argsort() now process data in smaller chunks to reduce temporary buffer memory usage.
- tmp_dst is allocated once before the processing loop instead of repeatedly.
- Chunked processing applied consistently across both CUDA CUB top-k and argsort implementations.
Upgrade notes
This is a CUDA-backend-only internal optimization with no CLI, API, or config changes; existing commands and flags (including the --output option from the prior build) continue to work. Rebuild or swap in the new binaries to benefit from lower temporary memory usage on CUDA devices.
Analyzed Oct 24, 2026
Upgrading?
Release notes summary
Optimized CUDA top-k and argsort implementations to reduce temporary buffer memory usage.
Key Highlights
- Process data in smaller chunks in CUDA top-k and argsort implementations
- Reduces temporary buffers memory usage significantly
- Allocates tmp_dst only once before the processing loop
- Uses chunked processing in both CUDA CUB top-k and argsort implementations
New Features
- CUDA memory optimization
- Reduced temporary buffer usage
Release Notes (excerpt)
<details open> ggml : process data in smaller chunks in CUDA ggml_top_k() and ggml_argsort() to reduce temporary buffers memory usage (#24776) * ggml : process data in smaller chunks in CUDA ggml_top_k() implementation to reduce temporary buffers memory usage * ggml : allocate tmp_dst only only once before the loop * chore : whitespaces Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * ggml : use chunked processing in both CUDA CUB top-k and argsort implementations…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system