b10199
ggml-org/llama.cppb10199Jul 30, 2026by github-actions[bot]same day as b10198
What changed since b10198
Server gains embedding-input support for next-token generation, plus a batch cleanup fix.
New features
- The HTTP server now accepts embedding inputs to drive next-token generation, extending how clients can seed sampling.
- Server-side handling of precomputed embeddings for sampled tokens is now supported alongside token-based input.
Improvements
- Destructor for server_batch() now properly releases resources, tightening server memory hygiene.
Bug fixes
- Fixed a resource leak in ~server_batch() where batch state was not fully cleaned up.
Upgrade notes
This is a routine incremental build with no config or migration steps required. Clients that previously could not submit embeddings to the server can now do so, but existing token-based request flows are unaffected. Rebuild/redeploy binaries as usual if you rely on the server component.
Analyzed Oct 31, 2026
Upgrading?
Release notes summary
The HTTP server has been updated to support embedding inputs for generating the next token, improving the server's input handling capabilities.
Key Highlights
- Added support for embedding inputs in the server to generate the next token
- Fixed resource cleanup in `server_batch()`
Release Notes (excerpt)
<details open> server: support inp embd to generate next token (#26313) * server: support embd for sampled token * fix ~server_batch() </details> **Website:** - <https://llama.app> **macOS/iOS:** - macOS Apple Silicon (arm64) - macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED - macOS Intel (x64) - iOS XCFramework **Linux:** - Ubuntu x64 (CPU) - Ubuntu arm64 (CPU) - Ubuntu s390x (CPU) - Ubuntu x64 (Vulkan) - Ubuntu arm64 (Vulkan) - Ubuntu x64 (ROCm 7.2) - Ubuntu x64 (OpenVINO)…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system