b9857

ggml-org/llama.cppb9857Jul 1, 2026by github-actions[bot]same day as b9856

What changed since b9856

low risk

Hexagon Flash Attention rework adds fp16 math, faster mask handling, and accuracy fixes

New features

  • Hexagon Flash Attention now supports FA_SELECT.
  • Hexagon kernels use fp16 math and vectorized sinks for faster attention.
  • Expanded tracing instrumentation covers all Hexagon FA functions with additional events.

Improvements

  • Mask processing optimized with fastdiv and better DMA caching, plus loop reordering to maximize cache hits.
  • Quantized matmul tasks folded into the main matmul threads and fused with ADD.
  • Kernel-params computation moved to the host and VTCM allocation refactored for consistency.
  • Aligned loads and uint32_t indices used throughout Hexagon FA loops.
  • KV and mask DMAs started earlier, and rows-per-task precomputed to avoid divisions.

Bug fixes

  • Hexagon FA accuracy improved through the kernel rework.
  • Fixed destination scratchpad alignment in the Hexagon matmul path.
  • Adjusted HVX fallback thresholds to recover token-generation regressions.

Upgrade notes

This release only touches the Hexagon backend; the CUDA FA changes from b9856 are unchanged, so no config or API migration is needed. Rebuild if you ship Hexagon binaries to pick up the new kernels and tracing. Other backends should behave identically to b9856.

Analyzed Oct 28, 2026

Upgrading?

Release notes summary

Performs a major rework of the Hexagon Flash Attention implementation, focusing on performance optimizations, accuracy improvements, and code refactoring including fp16 math usage and mask processing.

Key Highlights

  • Hexagon Flash Attention rework
  • Use of fp16 math in kernels
  • Optimized mask processing and caching
  • Vectorization of sinks and improved accuracy

Release Notes (excerpt)

<details open> hexagon: flash attention rework (optimizations, accuracy improvements, etc) (#25085) * hex-mm: fold mm quant tasks into the main matmul threads * hex-mm: minor formatting fixes * hex-mm: cleanup is_quant checks in dma dispatch * hex-mm: fix dst-spad alignment * hex-mm: move fp kernels in the hvx-mm-kernels header * hex-mm: fuse with ADD * hex-fa: factor out ukernels into separate headers and unify the rest * hex-fa: move kernel-params compute into the host…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories