b9966
ggml-org/llama.cppb9966Jul 11, 2026by github-actions[bot]same day as b9965
What changed since b9965
b9966 caches tensor-split regex patterns as static locals, eliminating per-call recompilation in decode threads.
Improvements
- Tensor-split regex patterns in llama_meta_device_get_split_state() are now compiled once as static const locals instead of 29 recompilations per call.
- Decode-thread overhead in -sm tensor mode is substantially reduced, since the callback previously ran once per tensor per token.
- The caching uses C++11 thread-safe local statics, so no locking or init changes are required.
Upgrade notes
This is a drop-in performance patch with no API, config, or CLI changes; the regex patterns are literal and stateless, so behavior is identical. Users running tensor-split mode (-sm tensor) should see the largest decode-speed gain. No migration steps needed.
Analyzed Oct 24, 2026
Upgrading?
Release notes summary
Optimized regex pattern compilation in `llama_meta_device_get_split_state()` by making patterns static const local variables to prevent recompilation on every call, significantly improving decode thread performance.
Key Highlights
- Performance optimization for tensor-split regex patterns
- Eliminated redundant regex compilation overhead in decode threads
- Thread-safe implementation using C++11 local statics
- Available across all supported platforms (macOS, Linux, Android, Windows, openEuler)
New Features
- Performance enhancement for tensor-split regex handling
Release Notes (excerpt)
<details open> llama : make tensor-split regex patterns static (#24710) llama_meta_device_get_split_state() recompiled 29 std::regex on every call. In -sm tensor mode the callback runs once per tensor per token, so this dominated the decode thread in profiling. Mark them static const so they are compiled once. Kept inside the function (local statics are thread-safe since C++11). Patterns are literal and stateless, so behavior is unchanged. </details> **macOS/iOS:**…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system