b9714

ggml-org/llama.cppb9714Jun 19, 2026by github-actions[bot]same day as b9713

What changed since b9713

low risk

b9714 adds an X-Accel-Buffering: no header to server streaming endpoints so Nginx reverse proxies stop buffering responses.

New features

  • Streaming server endpoints now emit an X-Accel-Buffering: no response header to disable Nginx proxy buffering.

Improvements

  • Streaming responses now work correctly behind Nginx reverse proxies, including clients like the Pi coding harness that previously stalled.

Bug fixes

  • Fixed broken streaming through Nginx caused by proxy response buffering on streaming endpoints.

Upgrade notes

No config or migration steps are needed; the header is applied automatically and only affects streaming endpoints. If you front llama.cpp with Nginx, you can drop any manual proxy_buffering off workaround for these routes. Non-streaming endpoints are unchanged.

Analyzed Oct 24, 2026

Upgrading?

Release notes summary

Adds a specific HTTP header to streaming endpoints to prevent Nginx reverse proxies from buffering responses, ensuring smooth streaming.

Key Highlights

  • Added 'X-Accel-Buffering: no' header to streaming endpoints
  • Fixes Nginx proxy buffering issues
  • Ensures compatibility with streaming applications

Release Notes (excerpt)

<details open> server: add "X-Accel-Buffering": "no" header to streaming endpoints (#24774) * server: add "X-Accel-Buffering": "no" header to streaming endpoints This header tells Nginx (as a reverse proxy) to NOT buffer responses. (only affects streaming endpoints) Without it, Nginx will break streaming with certain applications (notably the Pi coding harness). </details> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories