b10448

ggml-org/llama.cppb10448Aug 15, 2026by github-actions[bot]same day as b10447

What changed since b10447

medium risk

b10448 adds Kimi-K3 model support with latent MoE, situ activation, K3 chat format, and MXFP4 expert quantization.

Breaking changes

  • LLAMA_MAX_EXPERTS is raised to 1024, which can increase memory use and invalidate custom builds that hard-code the previous limit.

New features

  • Kimi-K3 text model support with hybrid KDA linear plus MLA full attention.
  • Architectural additions for K3: cross-layer residual attention, latent MoE, situ activation, MLA output gate, and full-rank KDA gate.
  • Dedicated K3 chat format including reasoning and tool-call handling.
  • MXFP4 support for routed experts, repacked losslessly from compressed-tensors mxfp4-pack-quantized weights.
  • Routing now keys off the top-level model name because K3's text_config reports the older KimiLinearForCausalLM architecture.

Improvements

  • MXFP4 repacking avoids a costly bf16 dequantize round-trip for large expert tensors.
  • KDA decay gate now supports the gate_lower_bound sigmoid variant used by K3 while leaving kimi-linear behavior unchanged.
  • Cross-layer residual weighted sums reuse the existing ggml_dsv4_hc_pre op instead of adding new kernels.

Upgrade notes

Rebuild from source since the expert-count constant and new model/quantization paths are compile-time changes; no config migration is needed for existing models. K3's cross-layer residual op is CPU and CUDA only, so Metal and Vulkan users should expect per-node fallback and slower inference until kernels land. The server thread-model redesign from b10447 is unchanged here, so no further server-side action is required.

Analyzed Oct 24, 2026

Upgrading?

Release notes summary

Major feature release adding support for the Kimi-K3 text model, which introduces novel architectural features like latent MoE, situ activation, and MLA output gates. It also includes a dedicated chat format for K3 and MXFP4 quantization support.

Key Highlights

  • Added Kimi-K3 text model with hybrid KDA + MLA attention
  • Introduced latent MoE and situ activation architecture
  • Added dedicated chat format for K3 with reasoning and tool-call support
  • Increased LLAMA_MAX_EXPERTS to 1024
  • Added MXFP4 quantization support for routed experts

Release Notes (excerpt)

<details open> model: add Kimi-K3 text model (#26185) * model: add Kimi-K3 text model Hybrid KDA (linear) + MLA (full) attention as in Kimi-Linear-48B, plus five things that architecture does not have: 1. cross-layer residual attention (attn_res_block_size) 2. latent MoE (routed experts run at n_expert_latent) 3. situ activation (replaces SwiGLU everywhere) 4. MLA output gate (sigmoid gate before o_proj)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories