Starstruk Logo MarkStarstruk

v0.15.0

actualbudget/actualv0.15.0May 1, 2026by jmartin-tech

AI Summary

This release expands the library's capabilities for LLM security testing by adding new probes, detectors, and generators, alongside documentation improvements and general bug fixes.

Key Highlights

  • Added 'Agent breaker' probe to test tools available to target systems.
  • Introduced 'ModernBERT refusal detector' for improved refusal detection capabilities.
  • Enhanced 'Bedrock generator' to support reasoning models.
  • Added 'NeMoGuardrails server support' as a new generator type.
  • Improved documentation structure and added build tests for documentation.

New Features

  • New probes: Multi-turn GOAT probe, Agent breaker, System prompt extraction probe, Homoglyph obfuscation prompts for smuggling probe.
  • New detectors: ModernBERT refusal detector.
  • New generators: NeMoGuardrails server support, Bedrock generator support for reasoning models.
  • Generators: Added mTLS client certificate authentication to REST generator.
  • General: Snapshot plugin cache into report JSONL, improved compatibility with older reports, fixed logic flip bug in encoding detection.

Full Release Notes

## What's Changed

### New plugins
* probes: Add multi-turn GOAT probe by @saichandrapandraju in https://github.com/NVIDIA/garak/pull/1424
* probes: Agent breaker by @eliyacohen-hub in https://github.com/NVIDIA/garak/pull/1628
* probes: Added system prompt extraction probe by @Nakul-Rajpal in https://github.com/NVIDIA/garak/pull/1538
* probes: add homoglyph obfuscation prompts to smuggling probe by @dentity007 in https://github.com/NVIDIA/garak/pull/1660
* detectors: Add ModernBERT refusal detector by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1650
* generators: NeMoGuardrails server support by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1675
* generators: test generator with reasoning traces by @leondz in https://github.com/NVIDIA/garak/pull/1697

### New features
* Agent breaker probe by @eliyacohen-hub in https://github.com/NVIDIA/garak/pull/1628 adds support for testing tools available to target systems. 

### Improved plugins
* generators: add mTLS client certificate authentication to REST generator by @JakeBx in https://github.com/NVIDIA/garak/pull/1681
* generators: Add support for reasoning models to Bedrock generator by @TheTeaCat in https://github.com/NVIDIA/garak/pull/1642
* detectors: limit tensor length passed to roberta by @leondz in https://github.com/NVIDIA/garak/pull/1715
* probes: Enforce soft_probe_prompt_cap in GCGCached probe by @precognitivem0nk in https://github.com/NVIDIA/garak/pull/1665
* detectors: fix logic flip bug in encoding detection, add tests to catch by @leondz in https://github.com/NVIDIA/garak/pull/1711

### Documentation
* docs: structure docs out of flat dir into tree by @leondz in https://github.com/NVIDIA/garak/pull/1662
* docs: explicitly enumerate configurable params in docs by @leondz in https://github.com/NVIDIA/garak/pull/1683

### Tuning & fixes
* fix: prevent semaphore leak by @r3v5 in https://github.com/NVIDIA/garak/pull/1657
* Replace base2048 with mikeshardmind-base2048 and add spec tests by @onceandtwice in https://github.com/NVIDIA/garak/pull/1518
* fix: remove duplicate status:2 attempt writes in TreeSearchProbe by @Christbowel in https://github.com/NVIDIA/garak/pull/1655
* task: dependency updates by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1678
* fix: report_digest -t taxonomy option not working by @HayatoFujihara in https://github.com/NVIDIA/garak/pull/1672
* fix: use magic.from_file() instead of from_buffer() for libmagic 5.47 compatibility by @HayatoFujihara in https://github.com/NVIDIA/garak/pull/1671
* housekeeping: prune phantom rapidfuzz dep by @leondz in https://github.com/NVIDIA/garak/pull/1701
* housekeeping: supply hook for signing, document signing prefs by @leondz in https://github.com/NVIDIA/garak/pull/1702
* first version of AGENTS.md by @leondz in https://github.com/NVIDIA/garak/pull/1698
* bug: avoid TBSA double horizontal rule by @leondz in https://github.com/NVIDIA/garak/pull/1712
* Handle unrecognized probes and detectors in report digest generation by @precognitivem0nk in https://github.com/NVIDIA/garak/pull/1663
* report: snapshot plugin cache into report jsonl by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1723
* docs: add build test for docs by @leondz in https://github.com/NVIDIA/garak/pull/1713
* improve compatibility with older reports by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1728

## New Contributors
* @r3v5 made their first contribution in https://github.com/NVIDIA/garak/pull/1657
* @onceandtwice made their first contribution in https://github.com/NVIDIA/garak/pull/1518
* @TheTeaCat made their first contribution in https://github.com/NVIDIA/garak/pull/1642
* @Christbowel made their first contribution in https://github.com/NVIDIA/garak/pull/1655
* @HayatoFujihara made their first contribution in https://github.com/NVIDIA/garak/pull/1672
* @precognitivem0nk made their first contribution in https://github.com/NVIDIA/garak/pull/1665
* @JakeBx made their first contribution in https://github.com/NVIDIA/garak/pull/1681
* @eliyacohen-hub made their first contribution in https://github.com/NVIDIA/garak/pull/1628
* @dentity007 made their first contribution in https://github.com/NVIDIA/garak/pull/1660

**Full Changelog**: https://github.com/NVIDIA/garak/compare/v0.14.1...v0.15.0