v2.6.1

p-e-w/hereticv2.6.1Aug 28, 2026by github-actions[bot]

Upgrading?

AI Summary

This release focuses on prompt rendering quality by implementing a unified chat-templating renderer using real GGUF templates and retiring the legacy standalone GPU server.

Key Highlights

  • Unified chat-templating renderer using real GGUF templates
  • Legacy server (`shimmy_server_gpu`) retired and removed (1,847 lines deleted)
  • Offline routing gate added for template validation
  • Updated Airframe dependency to version 0.4.1

Breaking Changes

  • Legacy standalone server `shimmy_server_gpu` removed

New Features

  • Real GGUF `chat_template` sourced from model metadata
  • shimmyjinja extensions for slice-with-step and Gemma family fallback
  • New `docs/CHAT_TEMPLATING.md` documentation

Full Release Notes

## [2.6.1] — 2026-08-28 — Airframe 0.4.1 and Unified Chat Templating

### Highlights

- **Single chat-templating renderer.** All paths (HTTP `/api/generate`, OpenAI
  `/v1/chat/completions` + `/v1/completions`, Anthropic `/v1/messages`, and the
  CLI `generate`) now render prompts through one `prompt_render` module that uses
  each model's **real GGUF `chat_template`** (Jinja via shimmyjinja), with a
  family fallback (ChatML/Llama3/Gemma) and a `--raw` escape for base/completion
  models. Previously instruct models were fed raw prompts or a coarse
  ChatML/Llama3/OpenChat heuristic — the root cause of garbled output on
  Qwen3/Gemma/etc.
- **Real GGUF template sourced at load.** `ModelSpec.chat_template` is populated
  from the GGUF header (airframe metadata), so Jinja-first rendering is automatic.
- **shimmyjinja extended** for real templates: slice-with-step (`messages[::-1]`,
  used by Qwen3 thinking) + Gemma family fallback.
- **Offline routing gate.** `tests/gguf_routing.rs` validates template routing for
  every model in the models dir (no GPU, ~14s) — the cheap per-model "screen test".
- **Legacy server retired.** `shimmy_server_gpu` (a dead standalone GPU server with
  its own template renderer) removed — 1,847 lines deleted. See `CHAT_TEMPLATING.md`.

### Docs

- New `docs/CHAT_TEMPLATING.md` — the canonical architecture reference.

### Release

- Updated the Airframe dependency to `0.4.1`.
- Published Linux x86_64, Windows x86_64, and macOS arm64 binaries.
- Corrected release artifact permissions and names so platform downloads are
  attached to the GitHub release.


Related Repositories