b10342
ggml-org/llama.cppb10342Aug 10, 2026by github-actions[bot]same day as b10338
What changed since b10338
Adds the Granite-Switch architecture with per-token LoRA adapter switching, on top of the prior MoE model-saver fix.
New features
- New `graniteswitch` architecture for dense all-attention Granite-4.1 models with embedded LoRA adapters.
- Per-token adapter selection driven by control tokens through a router attention layer.
- Conversion support for Granite 4.1 checkpoints, including stacking of multiple adapters into per-projection tensors.
- Working CPU and Apple Silicon Metal backends for the new architecture.
Improvements
- gguf-py schema extended with the new arch, KV keys, and stacked LoRA tensor naming plus writer helpers.
- Architecture registration and model loading wired into llama-arch and llama-model.
- Added a self-contained Metal build/convert/demo script for mid-sequence adapter switching.
Upgrade notes
No migration is required for existing models; the new architecture is additive. Be aware that adapter switch state is currently single-sequence only (proof-of-concept), so multi-sequence serving with Granite-Switch is not yet supported. Granite-Switch models must be converted with the updated conversion script to emit the switch metadata.
Analyzed Oct 29, 2026
Upgrading?
Release notes summary
Adds support for the new 'Granite-Switch' architecture, a dense all-attention model with LoRA adapters that can be selected per-token via control tokens using a router attention mechanism.
Key Highlights
- New `graniteswitch` architecture added to llama.cpp.
- Implements per-token adapter switching using a router attention layer.
- Supports conversion from Granite 4.1 checkpoints.
- Includes CPU and Metal (macOS) support.
Release Notes (excerpt)
<details open> model : Granite-Switch Architecture (#25107) * granite-switch: add llama.cpp backend (POC, CPU) New "granite-switch" architecture: a dense, all-attention Granite-4.1 model with N embedded LoRA adapters selected per-token by control tokens. - gguf-py schema (arch, KV keys, stacked LoRA tensor names) + writer helpers - conversion/granite.py: GraniteSwitchModel converter (stacks N adapters + zero base slot into per-projection A/B tensors; emits switch metadata)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system