Starstruk Logo MarkStarstruk

v0.17.1

NVIDIA/garakv0.17.1Oct 16, 2026by jmartin-tech

Upgrading?

What changed from v0.17.0 → v0.17.1?

AI Summary

This release introduces a new `hit@k` attack success rate metric and adds support for Python 3.14. It also includes numerous bug fixes for detectors, probes, and buffers to improve accuracy and stability.

Key Highlights

  • Added `hit@k` attack success rate metric for evaluating probe/detector pairs.
  • Added support for Python 3.14.
  • Fixed detector logic for Prefixes matching, safe-token suppression, and refusal counting.
  • Fixed buffer state management to prevent mutable state issues.
  • Updated documentation for CLI syntax and generator options.

New Features

  • hit@k attack success rate metric
  • Python 3.14 support

Full Release Notes

## What's Changed

### New features
* feat(evaluators): add hit@k attack success rate metric by @stefanoamorelli in https://github.com/NVIDIA/garak/pull/2090
  This new metric is currently exposed as terminal output and available per probe/detector pair in the final digest entry of the report.jsonl
* Add Python 3.14 support by @NK-Works in https://github.com/NVIDIA/garak/pull/2297

### Improved plugins
* fix(detectors): match mitigation.Prefixes at the start of the response by @chiruu12 in https://github.com/NVIDIA/garak/pull/2067
* fix(detectors): stop FileDetector scoring an unexamined file as a pass by @CTWalk in https://github.com/NVIDIA/garak/pull/2080
* fix(detectors): stop RefusalOnlyAdversarial over-counting skipped multi-turn attempts by @Yigtwxx in https://github.com/NVIDIA/garak/pull/2111
* fix(detectors): scope safe-token suppression to its own match by @feiiiiii5 in https://github.com/NVIDIA/garak/pull/2182
* fix(detectors): Scope ApiKey safe-token suppression to the matching credential by @MRX-72 in https://github.com/NVIDIA/garak/pull/2270
* fix(detectors): Do not treat octal 27 as a raw ANSI escape by @SashaMIT in https://github.com/NVIDIA/garak/pull/2255
* fix(probes): keep promptinject settings aligned with sampled prompts by @reeve25 in https://github.com/NVIDIA/garak/pull/2260
* fix(probes): restore DAN prompt compatibility by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/2282

### Documentation
* docs(rest): document updated CLI syntax and generator_options schema by @NK-Works in https://github.com/NVIDIA/garak/pull/2279

### Tuning & fixes
* fix(generators): raise clear error on HF InferenceAPI 404 response by @pujitha24 in https://github.com/NVIDIA/garak/pull/2112
* fix(buffs): use prompt setter in paraphrase buffs to update conversations by @bodapatisaikrishna in https://github.com/NVIDIA/garak/pull/2200
* fix: deep-copy mutable attempt state in Buff._derive_new_attempt by @pujitha24 in https://github.com/NVIDIA/garak/pull/2156
* fix(buffs): preserve attempt attributes and isolate mutable state in _derive_new_attempt by @bodapatisaikrishna in https://github.com/NVIDIA/garak/pull/2198
* fix: reuse the WebSocket event loop across calls by @Vamsi-2581 in https://github.com/NVIDIA/garak/pull/2269
* fix: respect parallel_capable for generator requests by @denelbjohn18 in https://github.com/NVIDIA/garak/pull/2274
* probes: factor up local prompt-cap logic into probes.base by @VishnuR23 in https://github.com/NVIDIA/garak/pull/2085
* task: update deps for security hygiene by @jmartin-tech in https://github.com/NVIDIA/garak/pull/2311
* lock transformers due to interface changes by @jmartin-tech in https://github.com/NVIDIA/garak/pull/2316
* Use --target_name as the WebSocket generator uri by @Somay-kousis in https://github.com/NVIDIA/garak/pull/2315
* report: update report probe tooltip text to be accurate (higher=better) by @leondz in https://github.com/NVIDIA/garak/pull/2317

## New Contributors
* @bodapatisaikrishna made their first contribution in https://github.com/NVIDIA/garak/pull/2200
* @pujitha24 made their first contribution in https://github.com/NVIDIA/garak/pull/2112
* @Vamsi-2581 made their first contribution in https://github.com/NVIDIA/garak/pull/2269
* @denelbjohn18 made their first contribution in https://github.com/NVIDIA/garak/pull/2274
* @MRX-72 made their first contribution in https://github.com/NVIDIA/garak/pull/2270
* @reeve25 made their first contribution in https://github.com/NVIDIA/garak/pull/2260
* @VishnuR23 made their first contribution in https://github.com/NVIDIA/garak/pull/2085
* @SashaMIT made their first contribution in https://github.com/NVIDIA/garak/pull/2255
* @NK-Works made their first contribution in https://github.com/NVIDIA/garak/pull/2297
* @chiruu12 made their first contribution in https://github.com/NVIDIA/garak/pull/2067
* @CTWalk made their first contribution in https://github.com/NVIDIA/garak/pull/2080
* @Yigtwxx made their first contribution in https://github.com/NVIDIA/garak/pull/2111
* @Somay-kousis made their first contribution in https://github.com/NVIDIA/garak/pull/2315

**Full Changelog**: https://github.com/NVIDIA/garak/compare/v0.17.0...v0.17.1

Related Repositories

public-apis/public-apis★ 491,192

A collective list of free APIs

NousResearch/hermes-agent★ 256,049

The agent that grows with you

yt-dlp/yt-dlp★ 200,221

A feature-rich command-line audio/video downloader

microsoft/markitdown★ 191,543

Python tool for converting files and office documents to Markdown.

Significant-Gravitas/AutoGPT★ 187,700

AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.