v1.0.0

ggml-org/llama.cppv1.0.0Nov 13, 2026by github-actions[bot]9 days after v0.9.0

Upgrading?

Release notes summary

llama.cpp v1.0.0 is a milestone release that prunes the deprecated public API and promotes the NextN/speculative decoding API into llama.h, alongside major inference features such as OCP FP8 KV caches, a host-memory MoE expert cache with a configurable CPU share, and external PLE/engram shard loading. It adds several new text and multimodal architectures (Kimi-K3.1-Pro, Ripple, xing4_0, drama-3 TTS, MageVL-Cowork, Kimi-K3.1 vision), hardens llama-server against SSRF/CORS/DNS-rebinding attacks, introduces an OpenAI-compatible /v1/audio/speech endpoint, and ships a redesigned web UI. The bundled ggml library is also bumped to v1.0.0 with broad cross-backend performance work.

Key Highlights

  • Public API cleanup: 35 deprecated functions, llama_set_warmup, and defrag_thold removed, while the NextN/speculative API is promoted to llama.h
  • New core capabilities: OCP FP8 KV caches with optional per-tensor scales, host-memory MoE expert cache with configurable CPU share and LRFU/ClassLRU replacement, and external PLE shard loading
  • New model support: Kimi-K3.1-Pro, Ripple-65B-Preview, xing4_0, drama-3 TTS, MageVL-Cowork, and Kimi-K3.1 vision
  • Server security hardening (SSRF, CORS, DNS rebinding, MCP tool cancellation) plus new /v1/audio/speech endpoint and reference-counted context checkpoints
  • Redesigned web UI with sidebar chat groups, live-preview markdown editor, native mobile layout, and a search_chats tool
  • ggml v1.0.0 with performance work across CUDA, Metal, Vulkan, SYCL, OpenCL, WebGPU, and Hexagon backends

Breaking Changes

  • Removal of llama_set_warmup and 35 deprecated public API functions from llama.h
  • Removal of defrag_thold from llama_context_params and the --defrag-thold CLI option
  • clip.h removed; model/subcontext infrastructure consolidated into mtmd
  • Non-fused GDN code paths and the ubatch-dependent graph node budget removed
  • ggml legacy DIAG_MASK ops and deprecated public API pruned in ggml v1.0.0
  • Server now requires explicit opt-in for unauthenticated tools (--allow-unauthenticated-tools) and private media URLs (--media-allow-private-urls)
  • HF_TOKEN usage restricted to huggingface.co hosts only

New Features

  • OCP FP8 KV cache support via -ctk/-ctv f8_e4m3 with optional static per-tensor scales
  • Host-memory MoE expert cache with configurable CPU share, LRFU/ClassLRU replacement, and no CPU wait on cached experts
  • External PLE/engram loading from separate GGUF shards via --shard/--ple
  • NextN/speculative API promoted to llama.h; llama_set_embd_state_src renamed to llama_set_ctx_state_src
  • New text architectures: Kimi-K3.1-Pro (DSA MLA), Ripple-65B-Preview (dense 65B), and xing4_0 (mHC/MLA/MoE/MTP)
  • drama-3 TTS (fishaudio) support with MTMD_GEN_AUDIO_TYPE_DRAMA3 and temperature control
  • MageVL-Cowork and Kimi-K3.1 vision model support
  • Gemma 4 streaming audio input and mtmd_helper_resampler for streaming raw PCM
  • OpenAI-compatible /v1/audio/speech endpoint and temperature option for /tts
  • Windows unbuffered model load mirroring the Linux DirectIO path
  • Backend scheduler and graph result reuse across reserves; unused graph metadata pages skipped
  • Server: reference-counted context checkpoints, parent-before-child slot visiting, and keep-awake during active requests
  • Server: MCP tool call cancellation on client disconnect or timeout
  • Server: top-level response_format.schema and Responses text.format JSON schema conversion
  • Jinja chained comparisons and argument unpacking; TTS text normalization via chat template
  • Web UI: sidebar folding/previewing/rearranging, named chat groups, sidebar footer navigation, native mobile layout, live-preview markdown editor, right-click actions, PDF thumbnails, and search_chats tool
  • ggml v1.0.0: configurable YaRN truncation, FP8 support, few-row quantized matmuls and MoE fusion on CUDA/Metal, Vulkan prefill FlashAttention, SYCL device-side MoE scheduler, Adreno FlashAttention in OpenCL, new WebGPU kernels, and expanded Hexagon op coverage

Release Notes (excerpt)

<img width="1600" height="900" alt="llama-cpp-v1" src="https://github.com/user-attachments/assets/ed94ed55-ebe2-4aee-bdfc-81bdaa3a3f59" /> Blog post: https://huggingface.co/blog/ggml-org/llama-cpp-v1 Overview llama.cpp v1.0.0 removes the deprecated public API and promotes the NextN/speculative API to `llama.h`, adds OCP FP8 KV caches, a host-memory MoE expert cache with a configurable CPU share, and loading of external PLE shards. It adds three text architectures (Kimi-K3.1-Pro, Ripple and…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories