Starstruk Logo MarkStarstruk

v0.15.1

FujiwaraChoki/MoneyPrinterV2v0.15.1Jun 5, 2026by jmartin-tech

AI Summary

This release introduces new ProPILE probes for PII leakage detection and support for the `simonw/llm` generator library. It also adds JSON and file output modes to the `qual_review` analysis module, alongside numerous bug fixes for generators, probes, and documentation updates.

Key Highlights

  • Added ProPILE probes for PII leakage detection
  • Added support for the `simonw/llm` generator library
  • Added JSON and file output modes to `garak.analyze.qual_review`
  • Fixed output handling for ggml generator and OpenAICompatible responses
  • Improved documentation and test stability

Breaking Changes

  • Removed deprecated maxrecall evaluator

New Features

  • ProPILE probes for PII leakage detection
  • simonw/llm generator support
  • JSON and file output modes in garak.analyze.qual_review

Full Release Notes

## What's Changed

### New plugins
* probes: add ProPILE probes for PII leakage detection by @stefanoamorelli in https://github.com/NVIDIA/garak/pull/1504
* generator: add `simonw/llm` library support by @Nakul-Rajpal in https://github.com/NVIDIA/garak/pull/1633

### New features
* Support JSON and file output modes in garak.analyze.qual_review  by @0x3df in https://github.com/NVIDIA/garak/pull/1641

### Improved plugins
* Fix ggml generator call output handling by @zw5 in https://github.com/NVIDIA/garak/pull/1733
* fix: handle None response.choices in OpenAICompatible by @Christbowel in https://github.com/NVIDIA/garak/pull/1654
* Handle local paths in Hugging Face file probe by @musaabhasan in https://github.com/NVIDIA/garak/pull/1740
* fix(detectors): keep packagehallucination entries with invalid dates by @ChrisJr404 in https://github.com/NVIDIA/garak/pull/1738
* Fix malformed MISP tag on snowball detectors by @adityasingh2400 in https://github.com/NVIDIA/garak/pull/1796
* fix(divergence): guard max_tokens override when the generator lacks it by @adityasingh2400 in https://github.com/NVIDIA/garak/pull/1795
* allow promptinject probes to limit `generation_params` by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1729
* fix: preserve all turns in NVMultimodal prompt prep by @fallintoplace in https://github.com/NVIDIA/garak/pull/1837

### Documentation
* housekeeping: amend arXiv `doc_uri`s from `html`  where available, else `abs` by @leondz in https://github.com/NVIDIA/garak/pull/1750
* Housekeeping: extend `AGENTS.md` by @leondz in https://github.com/NVIDIA/garak/pull/1771
* docs: Update contribution, agent instructions by @leondz in https://github.com/NVIDIA/garak/pull/1819

### Tuning & fixes
* chore: remove stale .orig merge artifact from detectors by @markd88 in https://github.com/NVIDIA/garak/pull/1731
* stabilize tests of reasoning lorem generation by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1741
* chore: remove deprecated maxrecall evaluator by @markd88 in https://github.com/NVIDIA/garak/pull/1732
* remove action workflow no longer used as the standard for the project by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1743
* update expectations for litellm proxy to openai by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1777
* fix: when tiktoken encoding is unclear fallback during estimation by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1749
* fix: improve data_path escape protections by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1781
* fix(evaluators): inline score_to_defcon in get_z_rating by @ppradyoth in https://github.com/NVIDIA/garak/pull/1794
* test(config): regression guard for config_files dedup by @ppradyoth in https://github.com/NVIDIA/garak/pull/1791
* test: add tests for evaluators by @r3v5 in https://github.com/NVIDIA/garak/pull/1656
* fix(docs): pin sphinx-rtd-theme>=3.1.0 for Sphinx 9.x compatibility by @AlexanderSanin in https://github.com/NVIDIA/garak/pull/1811
* fix: clearer error when probespec module has only inactive plugins by @notnick2 in https://github.com/NVIDIA/garak/pull/1797
* test: add unit tests for phrasing and sata probe modules by @markd88 in https://github.com/NVIDIA/garak/pull/1747
* Skip invalid payloads in _scan_payload_dir instead of crashing by @Kymi808 in https://github.com/NVIDIA/garak/pull/1810
* bump javascript dependencies by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1839


## New Contributors
* @markd88 made their first contribution in https://github.com/NVIDIA/garak/pull/1731
* @zw5 made their first contribution in https://github.com/NVIDIA/garak/pull/1733
* @musaabhasan made their first contribution in https://github.com/NVIDIA/garak/pull/1740
* @stefanoamorelli made their first contribution in https://github.com/NVIDIA/garak/pull/1504
* @ChrisJr404 made their first contribution in https://github.com/NVIDIA/garak/pull/1738
* @0x3df made their first contribution in https://github.com/NVIDIA/garak/pull/1641
* @adityasingh2400 made their first contribution in https://github.com/NVIDIA/garak/pull/1796
* @ppradyoth made their first contribution in https://github.com/NVIDIA/garak/pull/1794
* @AlexanderSanin made their first contribution in https://github.com/NVIDIA/garak/pull/1811
* @notnick2 made their first contribution in https://github.com/NVIDIA/garak/pull/1797
* @Kymi808 made their first contribution in https://github.com/NVIDIA/garak/pull/1810

**Full Changelog**: https://github.com/NVIDIA/garak/compare/v0.15.0...v0.15.1