b9626

ggml-org/llama.cppb9626Jun 13, 2026by github-actions[bot]same day as b9625

What changed since b9625

medium risk

Added Cohere2-MoE architecture support and fixed MTP issues.

Breaking changes

  • Renamed architecture from cohere2-moe to cohere2moe

New features

  • Added Cohere2-MoE (cohere2moe) architecture support
  • Added North-Mini-Code-1.0 model support

Improvements

  • Refactored tensor handling for expert selection and FFN lookup

Bug fixes

  • Fixed transformers crash related to 'first_k_dense_replace'
  • Fixed MTP failures and sliding window patterns

Upgrade notes

Update configuration files to use the new 'cohere2moe' architecture name. The MTP and sliding window fixes are applied automatically.

Analyzed Oct 22, 2026

Upgrading?

Release notes summary

Major update adding support for the Cohere2-MoE (Mixture of Experts) architecture, including specific tensor handling, tokenizer updates, and fixes for MTP and sliding windows.

Key Highlights

  • Added architecture support for Cohere2-MoE (renamed to cohere2moe)
  • Fixed transformers crash related to 'first_k_dense_replace'
  • Fixed MTP failures and sliding window patterns
  • Added support for 'North-Mini-Code-1.0' model
  • Added cohere2moe to Llama Model Saver supported list

Release Notes (excerpt)

<details open> Add arch support for cohere2-MoE (#24260) * Add arch support for cohere2-MoE * Removed redundant gating_func checks * Changed ffn lookup to prefer prefix_dense_intermediate_size * Renamed arch to cohere2moe * Removed redundant lmhead check and chat template changes * Removed lm_head.weight check from modify tensors, load output tensor not required, fallback to token_embd.weight * Changed to (routed+shared)*0.5 for shared expert combined avg…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories