b9974

ggml-org/llama.cppb9974Jul 12, 2026by github-actions[bot]same day as b9973

What changed since b9973

low risk

CUDA builds no longer crash when querying memory on devices with little or no free memory.

New features

  • CUDA builds now gracefully handle devices with insufficient memory instead of crashing at startup.
  • Explicitly disabled devices via the `-dev none` flag no longer cause hard crashes in CUDA builds.

Improvements

  • Memory query failures are now treated as non-fatal, reporting zero total/free memory for the affected device.
  • The layer-fit algorithm now naturally avoids placing layers on zero-memory devices, improving robustness on low-memory or unavailable GPUs.

Bug fixes

  • Fixed a fatal crash caused by cudaMemGetInfo() triggering an out-of-memory error on devices with no free memory.

Upgrade notes

This is a drop-in patch release with no config or API changes required. CUDA users on low-memory or multi-GPU systems, or those using `-dev none`, should upgrade to avoid startup crashes. Non-CUDA builds are unaffected.

Analyzed Oct 24, 2026

Upgrading?

Release notes summary

Prevents crashes in CUDA builds when querying memory on devices with insufficient memory. This allows the application to gracefully handle low-memory devices and explicitly disabled devices via the `-dev none` flag.

Key Highlights

  • CUDA memory query crash fix
  • Graceful handling of low-memory or unavailable CUDA devices
  • Support for explicit device disabling without hard crashes

Release Notes (excerpt)

<details open> cuda: Don't crash when querying memory on device with no free memory. (#25157) If a Cuda device has no or limited available memory, the actual call to cudaMemGetInfo() itself can cause a fatal crash due to a cuda out of memory error (there is not enough memory to actually query memory) This causes an issue because we query memory for all devices at startup even if the user isn't trying to use the device for inference.…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories