b10199

ggml-org/llama.cppb10199Jul 30, 2026by github-actions[bot]same day as b10198

What changed since b10198

medium risk

Server gains embedding-input support for next-token generation, plus a batch cleanup fix.

New features

  • The HTTP server now accepts embedding inputs to drive next-token generation, extending how clients can seed sampling.
  • Server-side handling of precomputed embeddings for sampled tokens is now supported alongside token-based input.

Improvements

  • Destructor for server_batch() now properly releases resources, tightening server memory hygiene.

Bug fixes

  • Fixed a resource leak in ~server_batch() where batch state was not fully cleaned up.

Upgrade notes

This is a routine incremental build with no config or migration steps required. Clients that previously could not submit embeddings to the server can now do so, but existing token-based request flows are unaffected. Rebuild/redeploy binaries as usual if you rely on the server component.

Analyzed Oct 31, 2026

Upgrading?

Release notes summary

The HTTP server has been updated to support embedding inputs for generating the next token, improving the server's input handling capabilities.

Key Highlights

  • Added support for embedding inputs in the server to generate the next token
  • Fixed resource cleanup in `server_batch()`

Release Notes (excerpt)

<details open> server: support inp embd to generate next token (#26313) * server: support embd for sampled token * fix ~server_batch() </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO)…

Published by ggml-org on GitHub.

Read the full release notes

Related Repositories