b10413

ggml-org/llama.cppb10413Aug 13, 2026by github-actions[bot]same day as b10412

What changed since b10412

low risk

Auto-detects speculative decoding type from local draft GGUF metadata when --spec-type is omitted.

New features

  • Loading a local draft model via -md without --spec-type now infers draft-dspark or draft-dflash from the GGUF's general.architecture and presence of a markov_w1.weight tensor.
  • Auto-detection emits an informational log line so users can see why speculative decoding was enabled.

Improvements

  • Spec-type detection logic was moved into the speculative module as common_speculative_types_from_gguf(), reusing RAII gguf_context_ptr for header parsing.
  • Previously, local draft models loaded into VRAM but speculative decoding silently stayed disabled; that silent-failure path is now resolved.
  • Added documentation covering the single-file assumption for split-GGUF draft models.

Bug fixes

  • Fixes the case where sidecar inference only checked HF repo sidecars and missed local draft files, leaving the spec type at NONE.

Upgrade notes

No config or CLI changes are required; existing --spec-type flags continue to take precedence. Users who previously had to pass --spec-type manually for local draft models can now omit it, but split-GGUF draft models may still need an explicit type since detection assumes a single file.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Adds automatic detection of speculative decoding types (dspark and dflash) when loading local draft models from GGUF metadata.

Key Highlights

  • Auto-detect spec type from draft GGUF metadata (#26814)
  • Add logging to inform users when auto-detection fires

Release Notes (excerpt)

<details open> common : auto-detect spec type from draft GGUF metadata (#26814) * common : auto-detect spec type from draft GGUF metadata When -md loads a local draft model without --spec-type, the sidecar inference in common_models_handler_apply only checks HF repo sidecars and misses local files. The draft model loads into VRAM but speculative decoding never activates (types stays NONE). Read general.architecture from the draft GGUF header and map:…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories