v0.9.0.6
NVIDIA/garakv0.9.0.6Jul 25, 2023by leondz
AI Summary
Introduces vulnerability reporting to AVID, package hallucination detection, and new CLI options for encoding payloads.
Key Highlights
- Integrated vulnerability reporting to AVID
- New package hallucination probe
- Primary/Extended detector designation
- Multiple encoding payload options (slurs, XSS)
Breaking Changes
- Removal of shortnames from probes and detectors
New Features
- AVID vulnerability reporting
- Package hallucination probe
- Primary/Extended detector designation
- Encoding payloads (slurs, xss)
Full Release Notes
### New in garak!
* **integrated vulnerability reporting:** vulnerabilities found with garak can now be directly reported to [AVID](https://avidml.org/) @shubhobm
* **package hallucination:** added a probe for detecting [package hallucination](https://vulcan.io/blog/ai-hallucinations-package-risk)
* **docs are up:** reference guide is here, https://reference.garak.ai/
* **primary/extended detectors:** it's now possible to designate a primary detector for a probe (when using the default probewise harness)
* **multiple payloads for encoding module:** as well as the default option, there's slurs and xss injection attempts; access them with `--probe_options '{"encoding.options": ["default", "slurs", "xss"]}'` (adjust to taste)
* **fine-tune perspective api backoff for bandwidth:** never wait sixty seconds, the window use to determine rate limit
* **doc fixes:** @mkonxd
* **hitlog entries now more self-contained:** store how many generations were targeted with that prompt
* **remove shortnames:** from probes and detectors
* **move encoding injection module to use triggers:** finer-grained detection, means fewer false positives