b10342

ggml-org/llama.cppb10342Aug 10, 2026by github-actions[bot]same day as b10338

What changed since b10338

medium risk

Adds the Granite-Switch architecture with per-token LoRA adapter switching, on top of the prior MoE model-saver fix.

New features

  • New `graniteswitch` architecture for dense all-attention Granite-4.1 models with embedded LoRA adapters.
  • Per-token adapter selection driven by control tokens through a router attention layer.
  • Conversion support for Granite 4.1 checkpoints, including stacking of multiple adapters into per-projection tensors.
  • Working CPU and Apple Silicon Metal backends for the new architecture.

Improvements

  • gguf-py schema extended with the new arch, KV keys, and stacked LoRA tensor naming plus writer helpers.
  • Architecture registration and model loading wired into llama-arch and llama-model.
  • Added a self-contained Metal build/convert/demo script for mid-sequence adapter switching.

Upgrade notes

No migration is required for existing models; the new architecture is additive. Be aware that adapter switch state is currently single-sequence only (proof-of-concept), so multi-sequence serving with Granite-Switch is not yet supported. Granite-Switch models must be converted with the updated conversion script to emit the switch metadata.

Analyzed Oct 29, 2026

Upgrading?

Release notes summary

Adds support for the new 'Granite-Switch' architecture, a dense all-attention model with LoRA adapters that can be selected per-token via control tokens using a router attention mechanism.

Key Highlights

  • New `graniteswitch` architecture added to llama.cpp.
  • Implements per-token adapter switching using a router attention layer.
  • Supports conversion from Granite 4.1 checkpoints.
  • Includes CPU and Metal (macOS) support.

Release Notes (excerpt)

<details open> model : Granite-Switch Architecture (#25107) * granite-switch: add llama.cpp backend (POC, CPU) New "granite-switch" architecture: a dense, all-attention Granite-4.1 model with N embedded LoRA adapters selected per-token by control tokens. - gguf-py schema (arch, KV keys, stacked LoRA tensor names) + writer helpers - conversion/granite.py: GraniteSwitchModel converter (stacks N adapters + zero base slot into per-projection A/B tensors; emits switch metadata)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories