llama.cpp Releases
684 releases of ggml-org/llama.cpp
- b10202
The SYCL backend has been optimized by fusing the RMS Normalization and Multiply operations into a single kernel.
Jul 31, 2026
- b10201
This release optimizes the WebGPU implementation of Flash Attention for quantized Key-Value (KV) caches, specifically improving performance at long context lengths.
Jul 31, 2026
- b10199
The HTTP server has been updated to support embedding inputs for generating the next token, improving the server's input handling capabilities.
Jul 30, 2026
- b10198
This release expands Vulkan backend capabilities by adding support for quantized concatenation operations.
Jul 30, 2026
- b10197
This release adds support for alternative convolution layouts in tests and fixes kernel storage requirements for the CPU backend to prevent test failures.
Jul 30, 2026
- b10196
This release addresses a context synchronization issue by ensuring pending asynchronous copies are completed before clearing embedding sequences.
Jul 30, 2026
- b10195
This release focuses on optimizing the build process for tests by removing redundant files and fixing quantization type selection logic. It removes the `get-model.cpp` file to prevent redundant builds and ensures correct quant type selection in test scenarios.
Jul 30, 2026
- b10194
Optimizes CUDA performance by allowing a transpose-free computation for matrices shaped 1xK using the mat_mul_vec_f function.
Jul 30, 2026
- b10192
A general synchronization commit for the ggml library, integrating changes across components.
Jul 30, 2026
- b10189
Refactors the M3 (Apple Silicon) graph by removing a custom CPU operation and implementing it using standard internal operations.
Jul 30, 2026
- b10188
Fixes a memory leak in the Metal backend where memory was not properly released if the model was freed before any GPU operations occurred.
Jul 30, 2026
- b10186
Fixes issues in the KleidiAI CI pipeline and resolves a compiler string overflow warning.
Jul 30, 2026
- b10184
Addresses review feedback for the MIMO2 (Multi-Input Multi-Output) feature.
Jul 30, 2026
- b10182
Refactors token suppression logic into the sampling module and addresses security vulnerabilities by removing the `has_logit_bias` flag.
Jul 29, 2026
- b10181
Fixes a crash on GPUs with less than 48 KiB of shared memory by disabling Multi-Matrix Quantization (MMQ) and falling back to BLAS.
Jul 29, 2026
- b10180
Optimizes the SYCL backend by implementing a contiguous fast path and using 32-bit index math for unary elementwise operations.
Jul 29, 2026
- b10179
Updates the BoringSSL vendor library to version 0.20260728.0 and disables KleidiAI builds for macOS and iOS.
Jul 29, 2026
- b10178
Enhanced server logging by adding trace logging for slot similarity checking during prompt cache slot selection, including details on skip reasons and calculations.
Jul 29, 2026
- b10176
Added a new RPC command `tensor_memset` to enable setting tensor memory values remotely.
Jul 29, 2026
- b10175
Added RDNA3.5 and RDNA3 to the multi-matrix quantization (mmq) configuration sets, allowing for independent tuning of these AMD architectures.
Jul 29, 2026