Starstruk Logo MarkStarstruk

v0.15.0

deepseek-ai/DeepSeek-OCR-2v0.15.0May 1, 2026by jmartin-tech

AI Summary

NVIDIA garak v0.15.0 adds new security probes and detectors including tool testing support, system prompt extraction, and ModernBERT refusal detection, alongside documentation restructuring.

Key Highlights

  • Multi-turn GOAT probe for advanced testing
  • Agent breaker probe for tool availability testing
  • ModernBERT refusal detector
  • NeMoGuardrails server support
  • Documentation restructuring into a tree structure

New Features

  • Multi-turn GOAT probe
  • Agent breaker probe (tool testing support)
  • System prompt extraction probe
  • Homoglyph obfuscation prompts
  • ModernBERT refusal detector
  • NeMoGuardrails server support
  • Reasoning traces test generator
  • mTLS client certificate authentication
  • Bedrock reasoning model support

Full Release Notes

## What's Changed

### New plugins
* probes: Add multi-turn GOAT probe by @saichandrapandraju in https://github.com/NVIDIA/garak/pull/1424
* probes: Agent breaker by @eliyacohen-hub in https://github.com/NVIDIA/garak/pull/1628
* probes: Added system prompt extraction probe by @Nakul-Rajpal in https://github.com/NVIDIA/garak/pull/1538
* probes: add homoglyph obfuscation prompts to smuggling probe by @dentity007 in https://github.com/NVIDIA/garak/pull/1660
* detectors: Add ModernBERT refusal detector by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1650
* generators: NeMoGuardrails server support by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1675
* generators: test generator with reasoning traces by @leondz in https://github.com/NVIDIA/garak/pull/1697

### New features
* Agent breaker probe by @eliyacohen-hub in https://github.com/NVIDIA/garak/pull/1628 adds support for testing tools available to target systems. 

### Improved plugins
* generators: add mTLS client certificate authentication to REST generator by @JakeBx in https://github.com/NVIDIA/garak/pull/1681
* generators: Add support for reasoning models to Bedrock generator by @TheTeaCat in https://github.com/NVIDIA/garak/pull/1642
* detectors: limit tensor length passed to roberta by @leondz in https://github.com/NVIDIA/garak/pull/1715
* probes: Enforce soft_probe_prompt_cap in GCGCached probe by @precognitivem0nk in https://github.com/NVIDIA/garak/pull/1665
* detectors: fix logic flip bug in encoding detection, add tests to catch by @leondz in https://github.com/NVIDIA/garak/pull/1711

### Documentation
* docs: structure docs out of flat dir into tree by @leondz in https://github.com/NVIDIA/garak/pull/1662
* docs: explicitly enumerate configurable params in docs by @leondz in https://github.com/NVIDIA/garak/pull/1683

### Tuning & fixes
* fix: prevent semaphore leak by @r3v5 in https://github.com/NVIDIA/garak/pull/1657
* Replace base2048 with mikeshardmind-base2048 and add spec tests by @onceandtwice in https://github.com/NVIDIA/garak/pull/1518
* fix: remove duplicate status:2 attempt writes in TreeSearchProbe by @Christbowel in https://github.com/NVIDIA/garak/pull/1655
* task: dependency updates by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1678
* fix: report_digest -t taxonomy option not working by @HayatoFujihara in https://github.com/NVIDIA/garak/pull/1672
* fix: use magic.from_file() instead of from_buffer() for libmagic 5.47 compatibility by @HayatoFujihara in https://github.com/NVIDIA/garak/pull/1671
* housekeeping: prune phantom rapidfuzz dep by @leondz in https://github.com/NVIDIA/garak/pull/1701
* housekeeping: supply hook for signing, document signing prefs by @leondz in https://github.com/NVIDIA/garak/pull/1702
* first version of AGENTS.md by @leondz in https://github.com/NVIDIA/garak/pull/1698
* bug: avoid TBSA double horizontal rule by @leondz in https://github.com/NVIDIA/garak/pull/1712
* Handle unrecognized probes and detectors in report digest generation by @precognitivem0nk in https://github.com/NVIDIA/garak/pull/1663
* report: snapshot plugin cache into report jsonl by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1723
* docs: add build test for docs by @leondz in https://github.com/NVIDIA/garak/pull/1713
* improve compatibility with older reports by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1728

## New Contributors
* @r3v5 made their first contribution in https://github.com/NVIDIA/garak/pull/1657
* @onceandtwice made their first contribution in https://github.com/NVIDIA/garak/pull/1518
* @TheTeaCat made their first contribution in https://github.com/NVIDIA/garak/pull/1642
* @Christbowel made their first contribution in https://github.com/NVIDIA/garak/pull/1655
* @HayatoFujihara made their first contribution in https://github.com/NVIDIA/garak/pull/1672
* @precognitivem0nk made their first contribution in https://github.com/NVIDIA/garak/pull/1665
* @JakeBx made their first contribution in https://github.com/NVIDIA/garak/pull/1681
* @eliyacohen-hub made their first contribution in https://github.com/NVIDIA/garak/pull/1628
* @dentity007 made their first contribution in https://github.com/NVIDIA/garak/pull/1660

**Full Changelog**: https://github.com/NVIDIA/garak/compare/v0.14.1...v0.15.0