b10242
ggml-org/llama.cppb10242Aug 3, 2026by github-actions[bot]same day as b10241
What changed since b10241
CUDA backend gains a penalties sampler (frequency, presence, top-k) with new tests and stricter penalty validation.
New features
- Backend sampler implementation for penalty sampling in the CUDA backend, covering frequency and presence penalties.
- Top-k penalty support added to backend sampling.
- New test utilities (accept_prompt, unique_prompt_tokens, compare_penalties_logits) and a backend penalties sampling test suite.
- Penalty sampler now inherits from llama_sampler_backend and accepts backend input handling.
Improvements
- Masked logits are preserved as -Inf instead of producing NaN, improving numerical stability of backend sampling.
- penalty_last_n now defaults from the model context when unspecified, and negative values for penalty_last_n and n_prev are clamped.
- Repeat penalty values are validated to be finite and greater than zero.
- Test helpers were refactored into shared functions for broader coverage.
Bug fixes
- Eliminated NaN generation in backend penalty sampling when logits are masked out.
- Guarded against invalid repeat-penalty values that previously could be passed through unchecked.
Upgrade notes
If you relied on an unset penalty_last_n, it now defaults from the model context rather than a fixed value, which can subtly change penalty behavior. Repeat penalty values must be finite and greater than zero; invalid configurations are now rejected, so audit custom sampler setups. The new backend sampler is CUDA-specific and opt-in via backend sampling paths.
Analyzed Nov 1, 2026
Upgrading?
Release notes summary
Implemented a backend sampler for handling penalty sampling logic (frequency, presence, and top-k) within the CUDA backend, accompanied by extensive test coverage.
Key Highlights
- Added backend sampler for penalty handling (frequency and presence)
- Added support for top-k penalties in backend sampling
- Enhanced numerical stability by preserving masked logits as -Inf
- Added comprehensive tests for backend penalties sampling
Release Notes (excerpt)
<details open> CUDA: Add backend sampler for penalties sampler (#25262) * sampling: enhance penalty handling in common_sampler_init - Set default value for penalty_last_n based on model context if not specified. - Ensure penalty_last_n and n_prev are non-negative. - Update llama_sampler_penalties structure to inherit from llama_sampler_backend and add backend input handling for penalties.…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system