b10142

<details open>

mtmd: Add Vision Support for Minimax-M3 (#25113)

* Add preliminary MiniMax-M3 support

Text-only port that re-uses existing components: MiniMax-M2 style GQA with
per-head QK-norm and partial rotary, DeepSeek-V3 style leading-dense and
routed/shared experts, and swigluoai activation. Sparse attention is not
yet supported (dense fallback); vision tower and MTP heads are dropped.

* MiniMax-M3 vision tower (mmproj + clip graph)

* Delete m3_vision_ref.py

* Update clip.cpp

* MSA

* Update constants.py

* Update minimax.py

* Cache creation. Working withotu flash attention

* Added flash attention for sparse layers

* Decomposed slow cpu OP into GPU + CPU ops. Massive speedup over long ctx

* Rewrote indexer op to be cuda native. Modified flash attention to match per group block picking

* Implement sparse attention calc out of stock ops.

* Fix a cache allocation and cont issue

* Fixed -fa auto crash, flagged debug spots

* Delete vocab.json

* Delete model.safetensors.index.json

* Delete generation_config.json

* Delete Minimax directory

* Handled multi stream case to fall back on Dense Attention

* Development scaffolding cleanup. No functional change to the decode or
4-way paths. Full debug harness remains at <8136a9c68ed7a5eb009aa67bba3fda8062f4648f> for reproducing the
selection-parity validation.

* Remove redundant comment from minimax-m3.cpp

* Changed 3 Gelu Ops for vision into Gelu_erf ops

* Assert that n_kv is multiple of 128

* Rename MSA index tensors to indexer convention

Note: All GGUFs generated before this change will need to be regenerated.

* Fix incorrect Assert

* Review driven changes (#3)

* Remove comment from conversion minimax.py

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* Remove whitespaces from constants.py

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* Tighten comment in minimax.py

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* inherit MiniMax-M3 from MiniMax-M2

* drop dead text_config fallbacks

* Add indexer writer methods

* Reuse LLM_FFN_SWIGLU_OAI_MOE

* Remove duplicate  indexer setters, add only block_size/local_blocks, follow value naming convention

* Fix conversion error /gguf_writer.py

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* Update gguf-py/gguf/gguf_writer.py

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* Update gguf-py/gguf/tensor_mapping.py

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* Update conversion/minimax.py

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* Update conversion/minimax.py

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* Remove whitespace in src/llama-kv-cache.cpp

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* Remove Whitespace in Update src/llama-model.h

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* Remove whitespace in src/llama-hparams.h

Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>

* Update minimax_m3.cpp

Rewrite code comment based on feedback and to better reflect the actual architecture, and reuse existing build_vit

* Rename minimax_m3.cpp to minimax-m3.cpp

* Update CMakeLists.txt

* Remove debug code from clip.cpp

* Update clip.cpp

* Update comments in tools/mtmd/models/minimax-m3.cpp

* Permute Q/K at conversion, drop precomputed sin/cos

* Log cache size on launch, block ctx shift, support prompt caching

Log indexer cache size on launch

Disallow ctx shift

Support prompt caching

* Update minimax-m3.cpp

* Optimize implementation, add multi stream support. 

Fully rewrote minimax-m3.cpp for speed and buffer size gains:

Unified the 4-way + decode, 1 FA call per layer instead of 4, with the groups mapped onto ne[3]

Custom CPU op now emits block-level mask, expanded on GPU, which causes CPU to GPU transfer to shrinks at prefill

Decode: ~25 nodes/layer vs ~50, no per-group concats/conts

Unified selection semantics, so both regimes rank bs + local bias (position-anchored local force), which means prefill/decode can no longer disagree on selection

can_reuse on the MSA bias input. Graph reuse at decode restored (was rebuilding the full graph every token)

In-place mask adds, shrinking compute buffer ~6.8 to ~4.2 GiB at ub2048/62k

Multi-stream: MSA now runs with -np N when kv_unified=false. Decode stays batched across streams (still 1 FA call), prefill loops per stream. dense fallback only for --kv-unified + multi-seq

Measured effect on expert offload bound setup: decode 6.2(4WAY)–7.15(MSA_decode) -> 7.7~7.8 t/s, flat from 5k to 60k+. prefill around 10% faster. buffer about 20% smaller, multi-user support.

* set default cache type to F32

* Fix potential DSA double indexer cache  allocation bug, only allocate in-cache k_idx for archs that opt in

* remove F16 downcasts in MSA attention, force F32 indexer score accum

* Add Minimax eos to llama vocab

* Guard edge case where idx cache can become stale after a tail trim

* Update llama-kv-cache.h

* Update llama-kv-cache.cpp

* Update llama-kv-cache.cpp

* Update llama-kv-cache.h

* Change resize Pad to none, resize alg to Bicubic Pillow

* Review driven changes

* Update llama-kv-cache.cpp

* rm unrotated pos_t

* fused rope w + pad

* rename merge --> merger for consistency

* add review skill for mtmd

* graph should use hparams n_merge

* fix lint

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Sigbjørn Skjæret <1629204+CISC@users.noreply.github.com>
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>

</details>

**Website:**
- <https://llama.app>

**macOS/iOS:**
- [macOS Apple Silicon (arm64)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-macos-arm64.tar.gz)
- macOS Apple Silicon (arm64, KleidiAI enabled) [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23780)
- [macOS Intel (x64)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-macos-x64.tar.gz)
- [iOS XCFramework](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-xcframework.zip)

**Linux:**
- [Ubuntu x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-ubuntu-x64.tar.gz)
- [Ubuntu arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-ubuntu-arm64.tar.gz)
- [Ubuntu s390x (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-ubuntu-s390x.tar.gz)
- [Ubuntu x64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-ubuntu-vulkan-x64.tar.gz)
- [Ubuntu arm64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-ubuntu-vulkan-arm64.tar.gz)
- [Ubuntu x64 (ROCm 7.2)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-ubuntu-rocm-7.2-x64.tar.gz)
- [Ubuntu x64 (OpenVINO)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-ubuntu-openvino-2026.2.1-x64.tar.gz)
- [Ubuntu x64 (SYCL FP32)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-ubuntu-sycl-fp32-x64.tar.gz)
- [Ubuntu x64 (SYCL FP16)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-ubuntu-sycl-fp16-x64.tar.gz)

**Android:**
- [Android arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-android-arm64.tar.gz)

**Windows:**
- [Windows x64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-win-cpu-x64.zip)
- [Windows arm64 (CPU)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-win-cpu-arm64.zip)
- [Windows arm64 (OpenCL Adreno)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-win-opencl-adreno-arm64.zip)
- [Windows x64 (CUDA 12)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-win-cuda-12.4-x64.zip) - [CUDA 12.4 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/b10142/cudart-llama-bin-win-cuda-12.4-x64.zip)
- [Windows x64 (CUDA 13)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-win-cuda-13.3-x64.zip) - [CUDA 13.3 DLLs](https://github.com/ggml-org/llama.cpp/releases/download/b10142/cudart-llama-bin-win-cuda-13.3-x64.zip)
- [Windows x64 (Vulkan)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-win-vulkan-x64.zip)
- [Windows x64 (OpenVINO)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-win-openvino-2026.2.1-x64.zip)
- [Windows x64 (SYCL)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-win-sycl-x64.zip)
- [Windows x64 (HIP)](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-bin-win-hip-radeon-x64.zip)

**openEuler:**
- [DISABLED](https://github.com/ggml-org/llama.cpp/pull/23705)
- openEuler x86 (310p)
- openEuler x86 (910b, ACL Graph)
- openEuler aarch64 (310p)
- openEuler aarch64 (910b, ACL Graph)

**UI:**
- [UI](https://github.com/ggml-org/llama.cpp/releases/download/b10142/llama-b10142-ui.tar.gz)