b9923

ggml-org/llama.cppb9923Jul 8, 2026by github-actions[bot]same day as b9922

What changed since b9922

medium risk

Server streaming internals refactored to PImpl with mutex-guarded state, renamed stream APIs, and quieter logging.

Breaking changes

  • Public stream functions are now scoped under a server_stream_ prefix, so any external code calling the old unprefixed names will fail to build.
  • The stream_read_status enum was moved out of the public header into the .cpp file, removing it from the exposed API surface.
  • The /v1/streams listing is gone, per the de-staled comments describing g_stream_sessions as internal-only.

Improvements

  • Session and manager state (done, completed_ts, GC running flag) are now plain members guarded by a mutex, with condvar predicates set under the lock.
  • The cancelled flag stays atomic so the lock-free should_stop poll is preserved.
  • Stream bring-up traces (skip drain, draining, DELETE evict, attach_pipe, router resume proxy) moved to debug level to cut default log noise.
  • Free functions are prefixed with server_stream_ to match the existing session manager start/stop naming.
  • The session manager is now a file-static singleton hidden in the .cpp, shrinking the public header and dev docs were updated accordingly.

Upgrade notes

If you embed or link against the server-stream headers, update call sites to the server_stream_ prefixed names and drop any references to the relocated enum or the removed /v1/streams listing. Stream bring-up messages no longer appear at default verbosity, so raise the log level if you depended on those traces for diagnostics. Pure binary users of the server need no config changes.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

Refactors server streaming implementation to use the PImpl pattern, improves thread safety with mutexes, and cleans up public headers.

Key Highlights

  • Refactored server-stream to PImpl pattern
  • Added mutex guards for session and manager state
  • Moved stream traces to debug level
  • Moved stream_read_status enum to the cpp file

Release Notes (excerpt)

<details open> server-stream: follow-up on SSE Replay Buffer (#23226) (#25047) * server-stream : pimpl * server-stream: prefix free functions with server_stream_ address review from ggerganov: scope the public stream functions under the server_stream_ prefix, matching server_stream_session_manager_start/stop. * server-stream: guard session and manager state with the mutex address review from ggerganov: make done, completed_ts and the GC running flag plain members under their…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories