Starstruk Logo MarkStarstruk

v0.15.0

Zackriya-Solutions/meetilyv0.15.0May 1, 2026by jmartin-tech

AI Summary

This release introduces new probes and detectors including NeMoGuardrails support, ModernBERT refusal detection, and tools for testing agent capabilities. It also includes improvements to the REST generator and documentation restructuring.

Key Highlights

  • Added new probes including Agent breaker, Multi-turn GOAT, and System prompt extraction.
  • Introduced ModernBERT refusal detector and NeMoGuardrails server support.
  • Enhanced REST generator with mTLS client certificate authentication.
  • Restructured documentation from a flat directory into a tree.

New Features

  • Agent breaker probe for testing tools available to target systems.
  • NeMoGuardrails server support.
  • Support for reasoning models in Bedrock generator.
  • mTLS client certificate authentication for REST generator.
  • Documentation restructuring.

Full Release Notes

## What's Changed

### New plugins
* probes: Add multi-turn GOAT probe by @saichandrapandraju in https://github.com/NVIDIA/garak/pull/1424
* probes: Agent breaker by @eliyacohen-hub in https://github.com/NVIDIA/garak/pull/1628
* probes: Added system prompt extraction probe by @Nakul-Rajpal in https://github.com/NVIDIA/garak/pull/1538
* probes: add homoglyph obfuscation prompts to smuggling probe by @dentity007 in https://github.com/NVIDIA/garak/pull/1660
* detectors: Add ModernBERT refusal detector by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1650
* generators: NeMoGuardrails server support by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1675
* generators: test generator with reasoning traces by @leondz in https://github.com/NVIDIA/garak/pull/1697

### New features
* Agent breaker probe by @eliyacohen-hub in https://github.com/NVIDIA/garak/pull/1628 adds support for testing tools available to target systems. 

### Improved plugins
* generators: add mTLS client certificate authentication to REST generator by @JakeBx in https://github.com/NVIDIA/garak/pull/1681
* generators: Add support for reasoning models to Bedrock generator by @TheTeaCat in https://github.com/NVIDIA/garak/pull/1642
* detectors: limit tensor length passed to roberta by @leondz in https://github.com/NVIDIA/garak/pull/1715
* probes: Enforce soft_probe_prompt_cap in GCGCached probe by @precognitivem0nk in https://github.com/NVIDIA/garak/pull/1665
* detectors: fix logic flip bug in encoding detection, add tests to catch by @leondz in https://github.com/NVIDIA/garak/pull/1711

### Documentation
* docs: structure docs out of flat dir into tree by @leondz in https://github.com/NVIDIA/garak/pull/1662
* docs: explicitly enumerate configurable params in docs by @leondz in https://github.com/NVIDIA/garak/pull/1683

### Tuning & fixes
* fix: prevent semaphore leak by @r3v5 in https://github.com/NVIDIA/garak/pull/1657
* Replace base2048 with mikeshardmind-base2048 and add spec tests by @onceandtwice in https://github.com/NVIDIA/garak/pull/1518
* fix: remove duplicate status:2 attempt writes in TreeSearchProbe by @Christbowel in https://github.com/NVIDIA/garak/pull/1655
* task: dependency updates by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1678
* fix: report_digest -t taxonomy option not working by @HayatoFujihara in https://github.com/NVIDIA/garak/pull/1672
* fix: use magic.from_file() instead of from_buffer() for libmagic 5.47 compatibility by @HayatoFujihara in https://github.com/NVIDIA/garak/pull/1671
* housekeeping: prune phantom rapidfuzz dep by @leondz in https://github.com/NVIDIA/garak/pull/1701
* housekeeping: supply hook for signing, document signing prefs by @leondz in https://github.com/NVIDIA/garak/pull/1702
* first version of AGENTS.md by @leondz in https://github.com/NVIDIA/garak/pull/1698
* bug: avoid TBSA double horizontal rule by @leondz in https://github.com/NVIDIA/garak/pull/1712
* Handle unrecognized probes and detectors in report digest generation by @precognitivem0nk in https://github.com/NVIDIA/garak/pull/1663
* report: snapshot plugin cache into report jsonl by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1723
* docs: add build test for docs by @leondz in https://github.com/NVIDIA/garak/pull/1713
* improve compatibility with older reports by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1728

## New Contributors
* @r3v5 made their first contribution in https://github.com/NVIDIA/garak/pull/1657
* @onceandtwice made their first contribution in https://github.com/NVIDIA/garak/pull/1518
* @TheTeaCat made their first contribution in https://github.com/NVIDIA/garak/pull/1642
* @Christbowel made their first contribution in https://github.com/NVIDIA/garak/pull/1655
* @HayatoFujihara made their first contribution in https://github.com/NVIDIA/garak/pull/1672
* @precognitivem0nk made their first contribution in https://github.com/NVIDIA/garak/pull/1665
* @JakeBx made their first contribution in https://github.com/NVIDIA/garak/pull/1681
* @eliyacohen-hub made their first contribution in https://github.com/NVIDIA/garak/pull/1628
* @dentity007 made their first contribution in https://github.com/NVIDIA/garak/pull/1660

**Full Changelog**: https://github.com/NVIDIA/garak/compare/v0.14.1...v0.15.0