b9856
ggml-org/llama.cppb9856Jul 1, 2026by github-actions[bot]same day as b9855
What changed since b9855
b9856 adds a CUDA Flash Attention refactor using __restrict__ pointers and PDL, on top of b9855's CPU NVFP4 AVX2 speedups.
New features
- CUDA Flash Attention kernels now integrate PDL (programmatic dependent launch) for improved kernel scheduling.
Improvements
- Consistent use of __restrict__ across CUDA Flash Attention code to help the compiler optimize memory access.
- Refactored FA kernels aim for better code consistency and optimization on NVIDIA GPUs.
- Retains the prior build's AVX2 NVFP4 dot-product path and UE4M3 lookup table for CPU inference.
Upgrade notes
This is a single-commit build bump with no API, config, or model-format changes, so a drop-in binary swap is sufficient. CUDA users may see modest Flash Attention performance differences; CPU users keep the NVFP4 AVX2 gains from b9855. No migration steps are required.
Analyzed Oct 31, 2026
Upgrading?
Release notes summary
Refactors CUDA Flash Attention code to consistently use pointer restrictions and PDL (Pointer Distance Logic) for better code optimization and consistency.
Key Highlights
- Consistent use of __restrict__ in CUDA
- PDL integration for Flash Attention
- Code optimization for FA kernels
Release Notes (excerpt)
<details open> CUDA: consistent use of __restrict__ + PDL for FA (#25185) </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32) - Ubuntu x64 (SYCL FP16) **Android:** - Android arm64 (CPU) **Windows:**…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system