b9827
ggml-org/llama.cppb9827Jun 27, 2026by github-actions[bot]same day as b9826
What changed since b9826
CUDA gains a cudaMemcpy2DAsync fast path for strided 2D copies, fixing multi-GPU GDN snapshot rollback.
New features
- CUDA ggml_cpy now has a fast path using cudaMemcpy2DAsync for same-type, same-shape 2D pitched block copies.
- New unit tests exercise the optimized strided copy path across backends.
Improvements
- Strided-but-row-contiguous tensors bypass the slow element-wise scalar copy kernel on CUDA, improving copy throughput.
- OpenVINO backend now explicitly reports strided copy as unsupported instead of failing the new tests.
Bug fixes
- Fixed GDN recurrent snapshot updates when running with multiple slots/GPUs (e.g. -np 4), where rollback slots are separated by cache stride gaps.
Upgrade notes
No config or API changes are required; this is a drop-in update from b9826. CUDA users with multi-slot or multi-GPU GDN workloads should see correctness and performance benefits. OpenVINO users relying on strided copies may hit an explicit unsupported-path error where the generic path was previously used.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
Optimized CUDA memory copy operations by adding a fast path for 2D strided tensor copies using cudaMemcpy2DAsync.
Key Highlights
- Added CUDA fast path for 2D strided copies
- Fixed GDN recurrent snapshot updates with multiple GPUs
- Added tests for the new optimized strided copy path
Release Notes (excerpt)
<details open> [CUDA] Added a cudaMemcpy2DAsync fast path to ggml_cuda_cpy (#25057) * [CUDA] Added a cudaMemcpy2DAsync fast path to ggml_cuda_cpy Add a CUDA ggml_cpy fast path for same-type, same-shape strided copies that are just 2D pitched block copies. When tensors are not fully contiguous but each row is contiguous, it now uses cudaMemcpy2DAsync instead of the slow element-wise scalar copy kernel.…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system