b9864

ggml-org/llama.cppb9864Jul 3, 2026by github-actions[bot]same day as b9862

What changed since b9862

low risk

Adds configurable SSE heartbeat pings so slow prefill no longer drops healthy stream connections

New features

  • New per-request sse_ping_interval body field lets clients control SSE heartbeat cadence, with -1 disabling pings.
  • Server now emits silent pings on idle SSE streams and only terminates a connection after 3s of observed silence.
  • WebUI sends sse_ping_interval of 1s in its request body to match its own 3s visibility-kick contract.

Improvements

  • Global default ping interval stays at 30s, so existing API clients see no behavioral change.
  • sse_ping_interval is now a typed schema field with range validation (-1 to INT32_MAX) instead of a raw JSON read.
  • Long-running prefill requests are less likely to be prematurely disconnected by intermediaries or the client.

Bug fixes

  • Fixes healthy SSE connections being dropped during slow prefill operations.

Upgrade notes

No configuration change is required: the server-wide default remains 30s and existing clients behave as before. Clients that need faster liveness signaling can set sse_ping_interval in the request body, using -1 to disable pings entirely. The CUDA gated_delta_net optimization from the previous build is unaffected by this change.

Analyzed Oct 30, 2026

Upgrading?

Release notes summary

Enhances server and UI stability for SSE streams by adding silent ping mechanisms. It introduces a configurable ping interval to prevent disconnects during slow prefill operations.

Key Highlights

  • New sse_ping_interval parameter defaults to 30s (WebUI uses 1s)
  • Kicks connections only after 3s of silence
  • Prevents dropping healthy connections during slow prefill
  • Server and UI updated to support per-request ping settings

Release Notes (excerpt)

<details open> server + ui: ping silent SSE streams every 1s and kick only after 3s so slow prefill never drops healthy connections (#25241) * server + ui: ping silent SSE streams every 1s and kick only after 3s so slow prefill never drops healthy connections * server + ui: sse_ping_interval becomes a per-request body field Address review from ngxson: the global default returns to 30 so API clients see no behavior change, and the WebUI sends sse_ping_interval: 1…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories