b9735

ggml-org/llama.cppb9735Jun 20, 2026by github-actions[bot]same day as b9733

What changed since b9733

low risk

b9735 adds an Intel AMX quantization optimization (up to ~1.47x faster prompt processing), with no other changes noted versus b9733.

Improvements

  • AMX quantization path reworked to flatten the partition across n_batch * M, so all threads contribute to the work.
  • Measured prompt-processing speedups on Intel Xeon Platinum 8488C ranging roughly 1.07x to 1.41x depending on quantization type.
  • Token-generation throughput stays essentially flat, so the gain is concentrated in prefill/quantization-heavy workloads.

Upgrade notes

This is a build-over-build patch with no config or API changes; the WebGPU F16 adapter toggles introduced in b9733 carry over unchanged. Intel Xeon AMX users should see faster prompt processing automatically, while other backends are unaffected. No migration steps are required.

Analyzed Nov 1, 2026

Upgrading?

Release notes summary

Optimized the AMX implementation for Intel Xeon processors to improve quantization performance.

Key Highlights

  • Optimized AMX implementation for Intel Xeon CPUs
  • Flattened partition over batch and M dimensions
  • Performance benchmarks show speedups up to 1.47x for quantization

New Features

  • Performance optimization for Intel AMX

Release Notes (excerpt)

<details open> ggml : optimize AMX (#24806) Flatten the partition over n_batch * M so every thread participates in the quantization | CPU | Model | Test | t/s OLD | t/s NEW | Speedup | |:--------------------------------|:------------------------------|:-------|----------:|----------:|----------:| | Intel(R) Xeon(R) Platinum 8488C | qwen35 0.8B IQ4_NL - 4.5 bpw | pp512 | 730.71 | 779.86 | 1.07 |…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories