b9951
ggml-org/llama.cppb9951Jul 10, 2026by github-actions[bot]same day as b9950
What changed since b9950
b9951 adds a brand-new ET backend with FlashAttention, SSM and optimized kernels, on top of b9950's test/build housekeeping.
Breaking changes
- The ET backend work includes ggml backend API changes that could affect out-of-tree backend implementations built against the previous interface.
New features
- Initial ET backend integrated into ggml, with kernels built as part of the ggml library.
- FlashAttention and SSM/Scan operator support in the new backend.
- Optimized kernels for MUL_MAT, MUL_MAT_ID, ROPE, RMS_NORM, GLU, SOFT_MAX, GET_ROWS, SET_ROWS and CONT.
- Backend documentation covering supported ops, plus performance logging.
- Cleaner CMake integration so the runtime is imported via find_package() instead of import hacks.
Improvements
- Kernel correctness and performance tuning across ROPE (yarn), SOFT_MAX (max_bias/sink), CONT and elmap paths.
- Vectorized fp32-to-fp16 conversion using custom instructions in SET_ROWS.
- Improved sinf implementation and logging helpers in the ET backend.
- Removed runtime import hacks in favor of standard package discovery.
Bug fixes
- Fixed GET_ROWS kernel losing the batch dimension.
- Removed an incorrect broadcasting guard in the SET_ROWS kernel.
- Corrected ROPE kernel behavior for yarn scaling.
- Fixed SOFT_MAX handling of max_bias and sink parameters.
- Fixed CONT to read with atomic stores instead of contiguous writes.
Upgrade notes
No configuration or model-format migration is required; the ET backend is a new, additive build target and existing CPU/CUDA/Vulkan/ROCm paths are unaffected. If you maintain a custom ggml backend, re-check it against the updated backend API introduced with the ET work. Rebuild from source rather than mixing binaries, since kernel code is now compiled into ggml itself.
Analyzed Oct 24, 2026
Upgrading?
Release notes summary
Introduces a new ET backend with FlashAttention and SSM support, along with optimized kernels for various operations like MUL_MAT, ROPE, and GLU.
Key Highlights
- Initial ET backend implementation
- FlashAttention and SSM support
- Optimized kernels for MUL_MAT, ROPE, GLU, and more
Release Notes (excerpt)
<details open> ggml-et: Initial ET backend (#24179) * ggml-et: Add performance logging * ggml-et: Quants helpers * ggml-et: Add MUL_MAT kernel * ggml-et: Add ROPE kernel * ggml-et: Add RMS_NORM kernel * ggml-et: Add GLU kernel * ggml-et: Add SOFT_MAX kernel * ggml-et: Add GET_ROWS kernel * ggml-et: Add CONT kernel * ggml-et: Add SET_ROWS kernel * ggml-et: Add MUL_MAT_ID kernel * ggml-et: Build et kernels as part of ggml * ggml-et: Embed kernels with fs fallback…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system