b9951

ggml-org/llama.cppb9951Jul 10, 2026by github-actions[bot]same day as b9950

What changed since b9950

medium risk

b9951 adds a brand-new ET backend with FlashAttention, SSM and optimized kernels, on top of b9950's test/build housekeeping.

Breaking changes

  • The ET backend work includes ggml backend API changes that could affect out-of-tree backend implementations built against the previous interface.

New features

  • Initial ET backend integrated into ggml, with kernels built as part of the ggml library.
  • FlashAttention and SSM/Scan operator support in the new backend.
  • Optimized kernels for MUL_MAT, MUL_MAT_ID, ROPE, RMS_NORM, GLU, SOFT_MAX, GET_ROWS, SET_ROWS and CONT.
  • Backend documentation covering supported ops, plus performance logging.
  • Cleaner CMake integration so the runtime is imported via find_package() instead of import hacks.

Improvements

  • Kernel correctness and performance tuning across ROPE (yarn), SOFT_MAX (max_bias/sink), CONT and elmap paths.
  • Vectorized fp32-to-fp16 conversion using custom instructions in SET_ROWS.
  • Improved sinf implementation and logging helpers in the ET backend.
  • Removed runtime import hacks in favor of standard package discovery.

Bug fixes

  • Fixed GET_ROWS kernel losing the batch dimension.
  • Removed an incorrect broadcasting guard in the SET_ROWS kernel.
  • Corrected ROPE kernel behavior for yarn scaling.
  • Fixed SOFT_MAX handling of max_bias and sink parameters.
  • Fixed CONT to read with atomic stores instead of contiguous writes.

Upgrade notes

No configuration or model-format migration is required; the ET backend is a new, additive build target and existing CPU/CUDA/Vulkan/ROCm paths are unaffected. If you maintain a custom ggml backend, re-check it against the updated backend API introduced with the ET work. Rebuild from source rather than mixing binaries, since kernel code is now compiled into ggml itself.

Analyzed Oct 24, 2026

Upgrading?

Release notes summary

Introduces a new ET backend with FlashAttention and SSM support, along with optimized kernels for various operations like MUL_MAT, ROPE, and GLU.

Key Highlights

  • Initial ET backend implementation
  • FlashAttention and SSM support
  • Optimized kernels for MUL_MAT, ROPE, GLU, and more

Release Notes (excerpt)

<details open> ggml-et: Initial ET backend (#24179) * ggml-et: Add performance logging * ggml-et: Quants helpers * ggml-et: Add MUL_MAT kernel * ggml-et: Add ROPE kernel * ggml-et: Add RMS_NORM kernel * ggml-et: Add GLU kernel * ggml-et: Add SOFT_MAX kernel * ggml-et: Add GET_ROWS kernel * ggml-et: Add CONT kernel * ggml-et: Add SET_ROWS kernel * ggml-et: Add MUL_MAT_ID kernel * ggml-et: Build et kernels as part of ggml * ggml-et: Embed kernels with fs fallback…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories