b10448
ggml-org/llama.cppb10448Aug 15, 2026by github-actions[bot]same day as b10447
What changed since b10447
b10448 adds Kimi-K3 model support with latent MoE, situ activation, K3 chat format, and MXFP4 expert quantization.
Breaking changes
- LLAMA_MAX_EXPERTS is raised to 1024, which can increase memory use and invalidate custom builds that hard-code the previous limit.
New features
- Kimi-K3 text model support with hybrid KDA linear plus MLA full attention.
- Architectural additions for K3: cross-layer residual attention, latent MoE, situ activation, MLA output gate, and full-rank KDA gate.
- Dedicated K3 chat format including reasoning and tool-call handling.
- MXFP4 support for routed experts, repacked losslessly from compressed-tensors mxfp4-pack-quantized weights.
- Routing now keys off the top-level model name because K3's text_config reports the older KimiLinearForCausalLM architecture.
Improvements
- MXFP4 repacking avoids a costly bf16 dequantize round-trip for large expert tensors.
- KDA decay gate now supports the gate_lower_bound sigmoid variant used by K3 while leaving kimi-linear behavior unchanged.
- Cross-layer residual weighted sums reuse the existing ggml_dsv4_hc_pre op instead of adding new kernels.
Upgrade notes
Rebuild from source since the expert-count constant and new model/quantization paths are compile-time changes; no config migration is needed for existing models. K3's cross-layer residual op is CPU and CUDA only, so Metal and Vulkan users should expect per-node fallback and slower inference until kernels land. The server thread-model redesign from b10447 is unchanged here, so no further server-side action is required.
Analyzed Oct 24, 2026
Upgrading?
Release notes summary
Major feature release adding support for the Kimi-K3 text model, which introduces novel architectural features like latent MoE, situ activation, and MLA output gates. It also includes a dedicated chat format for K3 and MXFP4 quantization support.
Key Highlights
- Added Kimi-K3 text model with hybrid KDA + MLA attention
- Introduced latent MoE and situ activation architecture
- Added dedicated chat format for K3 with reasoning and tool-call support
- Increased LLAMA_MAX_EXPERTS to 1024
- Added MXFP4 quantization support for routed experts
Release Notes (excerpt)
<details open> model: add Kimi-K3 text model (#26185) * model: add Kimi-K3 text model Hybrid KDA (linear) + MLA (full) attention as in Kimi-Linear-48B, plus five things that architecture does not have: 1. cross-layer residual attention (attn_res_block_size) 2. latent MoE (routed experts run at n_expert_latent) 3. situ activation (replaces SwiGLU everywhere) 4. MLA output gate (sigmoid gate before o_proj)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system