b10017

ggml-org/llama.cppb10017Jul 15, 2026by github-actions[bot]same day as b10016

What changed since b10016

low risk

b10017 only bumps the SYCL USM system-allocation threshold from 1 GiB to 4 GiB and adds debug logging, with no other changes.

Improvements

  • SYCL now uses a 4 GiB minimum buffer size for USM system allocations instead of 1 GiB, improving stability when model weights exceed available VRAM.
  • Added a debug message to make USM system allocations easier to trace.

Bug fixes

  • Mitigates failures when running models larger than device VRAM on SYCL hardware (e.g., Qwen3.5-35B-A3B-Q8 on a B70).

Upgrade notes

This is a SYCL-backend-only tweak; no config or API changes are required. Users on Intel GPUs running models that overcommit VRAM should see better stability, while other backends are unaffected. The Flash Attention XMX/oneDNN work from b10016 remains unchanged.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Adjusts SYCL memory allocation thresholds to handle larger models running on devices with limited VRAM, improving stability for systems with memory overcommitment.

Key Highlights

  • Increases minimum buffer size for USM system allocations from 1 GiB to 4 GiB
  • Addresses issues with models larger than available VRAM (e.g., Qwen3.5-35B-A3B-Q8 on B70)
  • Added debug logging for better tracking of USM system allocations

New Features

  • Increased USM buffer size threshold

Release Notes (excerpt)

<details open> sycl: Increase minimum buffer size for USM system allocations (#25525) Raise the threshold for minimum buffer size from 1 GiB to 4 GiB, based on real-world experiments of overcommitting device memory with model weights larger than available VRAM, for example Qwen3.5-35B-A3B-Q8 running on a B70. Also add a debug message to better track USM system allocations. Signed-off-by: Francois Dugast <francois.dugast@intel.com> </details> **macOS/iOS:** - macOS Apple Silicon (arm64)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories