v0.8.0

ggml-org/llama.cppv0.8.0Oct 28, 2026by github-actions[bot]13 days after v0.7.0

What changed since v0.7.0

high risk

v0.8.0 adds LFM3 audio/omni models, realtime/TTS endpoints, new architectures, and ggml v0.28.0 backend upgrades.

Breaking changes

  • DRY sampler now enforces a 64-breaker cap via LLAMA_DRY_MAX_SEQ_BREAKERS, which can change sampling for configs that previously used more distinct breakers.
  • ggml v0.28.0 introduces -1 expert-index semantics, so custom backend or model code relying on the previous expert-index convention may need updates.

New features

  • LiquidAI LFM3 family support covers text, vision, audio, and omni variants, including streaming audio input/output in mtmd.
  • New /v1/realtime and /tts server endpoints enable realtime audio and text-to-speech workflows.
  • Additional architectures include DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, Granite Switch Decision, and new decision/embedding models.
  • The web UI can browse and download models from Hugging Face Hub and search or compact conversations.
  • ggml v0.28.0 adds GGML_OP_TOP_K_MASK and broader backend capabilities.

Improvements

  • Sparse-attention QSA/DSA paths were reworked onto a fused masking operation.
  • Host-resident MoE experts now use a 50/50 SLRU cache.
  • Memory handling is cleaner with revised no_alloc behavior.
  • Server router memory management and tool-calling were improved.
  • Backend performance work spans CUDA chunked graph launches, SYCL graph replay, OpenCL Q2_K/Q3_K, Hexagon HTP, and KleidiAI SVE kernels.

Upgrade notes

Review DRY sampler configurations because distinct sequence breakers are now capped at 64, and validate any custom ggml/backend code against v0.28.0's -1 expert-index semantics. The v0.7.0 server changes (port 9931 and chat session API) remain in place, while the new audio endpoints are additive.

Analyzed Oct 29, 2026

Upgrading?

Release notes summary

v0.8.0 is a large release that introduces the LiquidAI LFM3 family (text, vision, audio and omni) alongside a new experimental streaming-audio API in mtmd and realtime audio support in the server via the new /v1/realtime and /tts endpoints. It adds many new architectures (DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, Granite Switch Decision, plus decision/embedding models), reworks sparse attention onto a fused masking op, adds a 50/50 SLRU cache for host-resident MoE experts, and improves router memory management, tool calling and the web UI. ggml is bumped to v0.28.0, bringing GGML_OP_TOP_K_MASK, -1 expert-index semantics and substantial backend performance work across CUDA, SYCL, OpenCL, Hexagon, Vulkan and KleidiAI.

Key Highlights

  • LiquidAI LFM3 family support: LFM3-1.7B (text), LFM3-VL-2B (vision), LFM3-Audio-2B and LFM3-Omni-2B with ASR, TTS and voice chat from a single model
  • New server endpoints: /v1/realtime implementing the OpenAI realtime protocol for conversation and transcription, plus POST /tts for text-to-speech models
  • Experimental mtmd_audio_stream API for streaming PCM audio input, used to support voxtral-realtime
  • Core improvements: fused QSA mask op for sparse attention, 50/50 SLRU cache for host-resident MoE experts, cleaner no_alloc memory handling and faster vocabulary loading
  • ggml bumped to v0.28.0 with GGML_OP_TOP_K_MASK, -1 expert-index semantics and backend work (CUDA chunked graph launches, SYCL graph replay, OpenCL Q2_K/Q3_K, Hexagon HTP rework, KleidiAI SVE kernels)

Release Notes (excerpt)

Overview This release adds the LiquidAI LFM3 family (text, vision, audio and omni) together with a new experimental streaming-audio API in `mtmd`, and wires realtime audio in and out of the server through the new `/v1/realtime` and `/tts` endpoints. A batch of new architectures landed: DeepSeek V4.1 Flash, SenseNova U1, Beam, Kolibri-1, Flint, Muse Spark, Granite Switch Decision, plus decision/embedding models from LiquidAI, Baidu and NVIDIA. The core got sparse-attention (QSA/DSA) reworked…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories