b10447

ggml-org/llama.cppb10447Aug 15, 2026by github-actions[bot]same day as b10446

What changed since b10446

medium risk

Server thread model redesigned so speculative processing runs in the worker thread, with worker and main thread roles swapped.

New features

  • Server's yield-to-queue handling now uses a redesigned thread model that moves speculative processing into the worker thread.

Improvements

  • Worker and main thread responsibilities were swapped to improve concurrency in the server's yield-to-queue path.
  • common_speculative_process now executes on the worker thread rather than the main thread.

Upgrade notes

This is an internal server concurrency refactor with no documented CLI, config, or API changes, so no migration steps are required. Because thread roles and speculative processing placement changed, re-run load and concurrency tests against your server deployment, especially if you rely on speculative decoding or yield-to-queue behavior.

Analyzed Oct 24, 2026

Upgrading?

Release notes summary

This release redesigns the server's yield-to-queue thread model by moving speculative processing into the worker thread and swapping the worker/main thread design for improved concurrency.

Key Highlights

  • Redesigned server thread model for yield-to-queue operations
  • Moved common_speculative_process to the worker thread
  • Swapped worker and main thread roles

Release Notes (excerpt)

<details open> server: re-design yield_to_queue thread model (#27133) * run common_speculative_process in worker * swap worker <--> main thread design </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.14)DISABLED…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories