llama.cpp Releases

Uniformizes the im2col destination type casting logic across all convolution operations to improve consistency and fixes casting logic to allow any kernel type.

Jul 14, 2026

Adds support for SME2 (ARMv9.2) f32 kernels with dynamic scheduling and releases standard binaries for multiple platforms including macOS, Linux, Android, Windows, and openEuler.

Jul 14, 2026

A fix to ensure log messages are flushed before the application exits when `usage()` is called, preventing potential message loss due to asynchronous logging from other threads.

Jul 14, 2026

Optimized the SYCL backend for Intel's Battlemage architecture by setting the vector thread count to 256, which is preferred by these chips, while keeping the default at 128 for other devices like ARC Alchemist.

Jul 14, 2026

Added support for the Q2_0 quantization format on the Metal backend (macOS/iOS). This allows for smaller model sizes and faster inference on Apple Silicon devices.

Jul 14, 2026

Added support for the Hy3 (Hunyuan 3) model architecture, a MoE decoder stack with specific routing mechanisms. It includes support for MTP (Multi-Token Prediction) speculative decoding for this specific model.

Jul 13, 2026

The Multi-MatMul-Q (MMQ) kernel configuration for CUDA was refactored to improve code organization and maintainability. This includes fixing the Blackwell configuration and removing legacy code to streamline the kernel management.

Jul 13, 2026

Added support for the Minimax2 EAGLE3 speculative decoding model, including a fix for a null pointer issue. This allows users to leverage faster inference speeds through speculative decoding with this specific model architecture.

Jul 13, 2026

A build system improvement to ensure tests use the shipped headers from the project rather than potentially system-installed ones, ensuring consistency and avoiding conflicts during compilation.

Jul 13, 2026

This release adds a new accessor function to the GGUF library to retrieve tensor shapes, improving the ability to inspect model metadata programmatically. It also refactors the accessor interface by removing the deprecated `n_dims` accessor.

Jul 13, 2026

A bug fix for the chat interface was implemented to prevent reasoning content from leaking when using force-opened bare templates, specifically addressing a regression with Nemotron Nano v2.

Jul 13, 2026

This release introduces fused top-k Mixture-of-Experts (MoE) support for the SYCL backend, optimizing performance for specific hardware configurations. It also includes various binary builds across different platforms (macOS, Linux, Windows, Android) and architectures.

Jul 13, 2026

Adds support for Q2_K quantization format to the SYCL (Intel oneAPI) DMMV (Dynamic Multi-Vector) reordering path, improving performance and compatibility for Intel hardware.

Jul 13, 2026

Fixes a bug where per-request overrides for `reasoning_budget_tokens` and `reasoning_budget_message` in chat completions were ignored in favor of server defaults, breaking support for advanced reasoning models like Qwen3.

Jul 13, 2026

Updates the embedded HTTP server library (cpp-httplib) to version 0.50.1, incorporating the latest fixes and improvements from the dependency.

Jul 13, 2026

Fixes the `/v1/models` endpoint to correctly report input modalities, ensuring that models with multimodal projection explicitly disabled via `--no-mmproj-auto` are not falsely advertised as multimodal.

Jul 13, 2026

Fixes a silent prompt truncation bug where prompts containing embedded NUL bytes were cut off at the tokenizer boundary, causing subsequent messages and markers to be dropped.

Jul 12, 2026

Improves checkpoint management in the server by evicting checkpoints that are too close in step count, helping to optimize storage usage and prevent accumulation of redundant files.

Jul 12, 2026

Fixes a bug where image blocks in tool results were silently dropped during the Anthropic-to-OpenAI conversion process, breaking multimodal tool outputs.

Jul 12, 2026

Fixes the conditional logic for displaying an error message regarding unsupported tensor split modes on specific architectures, ensuring users receive appropriate feedback.

Jul 12, 2026