llama.cpp Releases

Optimizes CUDA decoding performance by tuning the crossover point between mul_mat_vec_q and MMQ batch kernels, adding hardware-specific switch points to handle different GPUs and quantization types.

Aug 21, 2026

Improves robustness in JSON schema parsing by gracefully falling back when unsupported regex patterns are encountered, preventing application crashes.

Aug 21, 2026

Optimizes Flash Attention on the Metal backend by dequantizing KV cache to F16 in a preprocessing pass, improving accuracy and performance for various quantized KV cache types.

Aug 21, 2026

Reverts a previous fix for the tensor-split meta backend to address issues introduced by the change, restoring the previous behavior.

Aug 21, 2026

Fixes a compilation issue on 32-bit ARM architectures by gating the use of `__fp16` on the IEEE half format, ensuring compatibility with toolchains that provide the type but use a different format.

Aug 20, 2026

Refactors the server's sleep handling logic to allow access to the `/metrics` endpoint even when the server is in a sleeping state, improving observability and fixing race conditions.

Aug 20, 2026

Optimizes Flash Attention in the Vulkan backend by dequantizing KV cache only once in cooperative matrix operations and handling edge cases like FA scratch buffer limits.

Aug 20, 2026

Addresses a potential crash in the Vulkan backend by adding null checks in the queue command pools cleanup function.

Aug 20, 2026

Introduces support for new Granite model architectures featuring Sliding Window Attention (SWA), including GraniteSWAForCausalLM and GraniteMoeSWAForCausalLM, along with infrastructure for converting rope patterns.

Aug 20, 2026

Adds a new parameter `ggml_rope_set_offset` to control Rotary Positional Embedding (RoPE) offsets, with full support implemented for CPU, Metal, CUDA, and Vulkan backends.

Aug 20, 2026

This release introduces a minor update to the Multi-Modal (mtmd) module, specifically adding functionality for bitmap set merging to improve memory management.

Aug 20, 2026

Optimizes the macOS Metal backend by implementing q8_0 dequantization using packed types, which should improve performance on Apple Silicon.

Aug 20, 2026

Introduces a new server preset option to deduplicate cached models, allowing for more efficient memory usage and cache management.

Aug 20, 2026

Adds CI attestation capabilities to verify the integrity and origin of signed release artifacts, enhancing security and trust in the build process.

Aug 19, 2026

Upgrades the OpenVINO backend to version 2026.3 and updates device drivers, while skipping a specific CI rollback test for the Nemotron-H model.

Aug 18, 2026

Resolves a bug in the multi-modal model (mtmd) where the LFM2 image tiling threshold was incorrectly processed, ensuring compatibility across platforms.

Aug 18, 2026

Synchronizes core ggml library code with the main repository, incorporating necessary updates from the upstream source.

Aug 18, 2026

Improves build system robustness by fixing xcframework generation, removing unused include paths, and standardizing vendored library targets using CMake aliases.

Aug 18, 2026

Fixes memory reporting on AMD APUs within HIP builds to prevent over-promising available memory on systems with small memory carve-outs.

Aug 17, 2026

This release updates the CI/CD pipeline to explicitly push the release tag using git tag before creating the release via the GitHub API, ensuring the tag is available for workflows.

Aug 17, 2026