b9626
ggml-org/llama.cppb9626Jun 13, 2026by github-actions[bot]same day as b9625
What changed since b9625
Added Cohere2-MoE architecture support and fixed MTP issues.
Breaking changes
- Renamed architecture from cohere2-moe to cohere2moe
New features
- Added Cohere2-MoE (cohere2moe) architecture support
- Added North-Mini-Code-1.0 model support
Improvements
- Refactored tensor handling for expert selection and FFN lookup
Bug fixes
- Fixed transformers crash related to 'first_k_dense_replace'
- Fixed MTP failures and sliding window patterns
Upgrade notes
Update configuration files to use the new 'cohere2moe' architecture name. The MTP and sliding window fixes are applied automatically.
Analyzed Oct 22, 2026
Upgrading?
Release notes summary
Major update adding support for the Cohere2-MoE (Mixture of Experts) architecture, including specific tensor handling, tokenizer updates, and fixes for MTP and sliding windows.
Key Highlights
- Added architecture support for Cohere2-MoE (renamed to cohere2moe)
- Fixed transformers crash related to 'first_k_dense_replace'
- Fixed MTP failures and sliding window patterns
- Added support for 'North-Mini-Code-1.0' model
- Added cohere2moe to Llama Model Saver supported list
Release Notes (excerpt)
<details open> Add arch support for cohere2-MoE (#24260) * Add arch support for cohere2-MoE * Removed redundant gating_func checks * Changed ffn lookup to prefer prefix_dense_intermediate_size * Renamed arch to cohere2moe * Removed redundant lmhead check and chat template changes * Removed lm_head.weight check from modify tensors, load output tensor not required, fallback to token_embd.weight * Changed to (routed+shared)*0.5 for shared expert combined avg…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system