llama.cpp Releases

Extends the Metal backend concat operator to support a wider range of tensor types including f16, bf16, i8, i16, and i64.

Jun 17, 2026

Expands the Server Router with a comprehensive model management API, including endpoints for downloading, deleting models, and real-time SSE updates.

Jun 17, 2026

Modifies validation logic to skip main GPU configuration checks when no devices are available, improving handling of empty device scenarios.

Jun 17, 2026

Fixes a segmentation fault that occurred when processing long prompts with the Eagle3 model specification.

Jun 17, 2026

Adds device-to-device memory copy functionality using the SYCL API. This release also updates the runtime table and fixes P2P communication detection methods.

Jun 17, 2026

Introduces a new 3D convolution operator for the SYCL backend, accompanied by optimization updates and documentation changes.

Jun 17, 2026

This release enhances the Vulkan backend by recording actual memory properties during buffer creation. This allows for better memory management and selection of optimal memory types.

Jun 17, 2026

This release fixes continuous integration issues related to Vulkan Docker images. Updates to the `vulkan-shaders-gen.cpp` file address potential undefined behavior and update shader generation logic.

Jun 17, 2026

This release optimizes the OpenCL backend's matrix multiplication operation (`mul_mat_f16_f32_l4`) specifically for the decode phase, likely improving performance for inference tasks.

Jun 17, 2026

This release improves the logging system in the common library by enforcing a maximum capacity and optimizing queue resizing logic. This enhances performance and control over logging operations.

Jun 17, 2026

This release extends the SYCL backend's support for FP16 (half-precision) floating-point operations by adding support for mathematical functions like square, square root, logarithm, sine, cosine, and clamp.

Jun 17, 2026

This release fixes a critical use-after-free bug in the SYCL backend related to asynchronous memory copy operations during Mixture-of-Experts (MoE) prefill phases. This ensures memory safety and stability when running MoE models on SYCL devices.

Jun 17, 2026

This release introduces optional Unified Shared Memory (USM) system allocations for the SYCL backend, allowing for VRAM overcommit on devices with limited memory. The feature is disabled by default and can be enabled via the `GGML_SYCL_USM_SYSTEM` environment variable.

Jun 17, 2026

This release updates the internal BoringSSL library to version 0.20260616.0, improving cryptographic functionality and security. Standard binary builds for major platforms are provided alongside this update.

Jun 16, 2026

A bug fix release addressing edge cases in NVFP4 (NVIDIA Floating Point 4) quantization within the llama-graph implementation. The changes ensure correct order of operations for LoRA and ModelOPT, specifically moving post-GEMM MUL operations.

Jun 16, 2026

This release adds backend sampling support for the Eagle3 model, enhancing its integration capabilities. It provides pre-compiled binaries for macOS, Linux, Windows, Android, and openEuler across various hardware architectures and accelerators.

Jun 16, 2026

Optimizes Vulkan performance on Unified Memory Architecture (UMA) devices by implementing a preference for host-visible memory buffers to improve memory management efficiency.

Jun 16, 2026

Extends Vulkan backend support to include the gated_delta_net activation function with a specific state vector size of 16.

Jun 16, 2026

Adds a new command-line flag to the benchmarking tool, enabling users to run benchmarks without requiring network connectivity.

Jun 16, 2026

Enhances SYCL support for Mixture of Experts (MoE) models by extending reordered-weight handling to Q4_K, Q5_K, and Q6_K quantization formats during matrix multiplication (MUL_MAT_ID) and DMMV operations.

Jun 16, 2026