b10369
ggml-org/llama.cppb10369Aug 12, 2026by github-actions[bot]same day as b10362
What changed since b10362
b10369 adds Pocket-TTS audio generation to MTMD with a large convolution speedup, while b10362 was only test/CI housekeeping.
New features
- Multi-modal (MTMD) now supports Pocket-TTS text-to-speech models.
- New audio generation parameters exposed: frames_after_eos, pad_short_text, and model_variant.
- Per-language-pack settings (end-of-speech padding, short-prompt padding, temperature) are now carried in the mmproj metadata.
Improvements
- Transposed convolutions in the Pocket-TTS decoder are rebuilt as GEMM plus col2im, collapsing many small graph nodes into fewer kernel launches.
- Per-frame generation latency drops roughly 80% on CUDA and 50% on CPU.
- Output is numerically equivalent to the previous implementation (correlation ~0.999994, identical frame counts).
- Streaming state, overlap-add tail, and bias handling are unchanged by the convolution rewrite.
Bug fixes
- Depthwise upsampling no longer floods the graph with per-channel convolutions and concats, removing a launch-bound bottleneck.
Upgrade notes
Existing text and vision workflows are unaffected; the changes are additive to the MTMD audio path. To use Pocket-TTS you need an mmproj that includes the new clip.gen.audio.frames_after_eos and clip.gen.audio.pad_short_text keys, so regenerate or re-download the matching projector rather than reusing an older one. No config migration is required for non-audio users.
Analyzed Oct 28, 2026
Upgrading?
Release notes summary
Added support for Pocket-TTS in Multi-Modal models, optimizing convolution operations to reduce generation latency significantly.
Key Highlights
- Added Pocket-TTS support for MTMD
- Optimized transposed convolutions as GEMM + col2im
- Generation time reduced by 80% on CUDA and 50% on CPU
- Added new audio generation parameters (frames_after_eos, pad_short_text, model_variant)
Release Notes (excerpt)
<details open> mtmd: support pocket-tts (#26871) * adapt the api * text model ok * working impl, need verify and clean up * mtmd: build the pocket-tts transposed convolutions as GEMM + col2im ggml_conv_transpose_1d has no grouped mode, so the depthwise upsample was built as one convolution and one concat per channel, which floods the graph with small nodes and makes kernel launches dominate the decoder. Fold both cases into the column form the seanet decoder already needs:…
Published by ggml-org on GitHub.
Read the full release notesRelated Repositories
:electron: Build cross-platform desktop apps with JavaScript, HTML, and CSS
Godot Engine – Multi-platform 2D and 3D game engine
ClickHouse® is a real-time analytics database management system
Self-hosted game stream host for Moonlight.
DuckDB is a high-performance analytical database system