b9847

ggml-org/llama.cppb9847Jun 30, 2026by github-actions[bot]same day as b9846

What changed since b9846

low risk

b9847 adds a CUDA FlashAttention fix for Gemma E4B MTP; the Vulkan Asahi matmul change from b9846 carries over unchanged.

Bug fixes

  • Corrected FlashAttention computation for the Gemma E4B (4B) model with multi-token prediction on the CUDA backend.
  • Removed an unused template declaration as part of the CUDA fix cleanup.

Upgrade notes

This is a single-commit follow-up to b9846 with no config or API changes, so a straight binary swap is sufficient. CUDA users running Gemma E4B with MTP should upgrade to get correct FlashAttention output; other backends see no functional difference.

Analyzed Oct 30, 2026

Upgrading?

Release notes summary

This release fixes a critical bug in the FlashAttention implementation for the Gemma E4B (4 Billion parameter) model with MTP (Multi-Token Prediction) on the CUDA backend. Standard binaries are included for all supported platforms.

Key Highlights

  • Fixes FlashAttention for Gemma E4B MTP on CUDA
  • Includes standard binaries for macOS, Linux, Android, Windows, and openEuler
  • KleidiAI support remains disabled

New Features

  • CUDA Gemma E4B MTP FlashAttention fix

Release Notes (excerpt)

<details open> CUDA: fix Gemma E4B MTP FlashAttention (#25148) * CUDA: fix Gemma E4B MTP FlashAttention * remove unused template declaration </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO) - Ubuntu x64 (SYCL FP32)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories