Starstruk Logo MarkStarstruk

v0.14.1

Zackriya-Solutions/meetilyv0.14.1Apr 3, 2026by jmartin-tech

AI Summary

This release introduces a WebSocket generator for real-time LLM security testing, supports OpenAI Harmony format for audio inputs, and enhances detection capabilities for various injection attacks. It also includes significant cleanup by removing deprecated nemollm dependencies.

Key Highlights

  • New WebSocket generator for real-time LLM security testing
  • OpenAI Harmony format support for audio inputs
  • Breaking change: Removal of deprecated nemollm generator and dependencies
  • Improved Jinja Injection Detection
  • Added Bootstrap Confidence Intervals for Attack Success Rates

Breaking Changes

  • remove deprecated nemollm generator and dependencies

New Features

  • OpenAI Harmony format for audio inputs
  • Add probe tier to --list_probes
  • Add Bootstrap Confidence Intervals for Attack Success Rates
  • New WebSocket generator for real-time LLM security testing

Full Release Notes

## What's Changed

### New plugins
* generator: Add WebSocket generator for real-time LLM security testing by @dyrtyData in https://github.com/NVIDIA/garak/pull/1379

### New features
* OpenAI Harmony format for audio inputs by @erickgalinkin in https://github.com/NVIDIA/garak/pull/1618
* add probe tier to `--list_probes` by @leoneperdigao in https://github.com/NVIDIA/garak/pull/1571
* Add Bootstrap Confidence Intervals for Attack Success Rates by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1577

#### Notice: Breaking changes in this release
* remove deprecated nemollm generator and dependencies by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1643

### Improved plugins
* probes: activate sata & badchars by @leondz in https://github.com/NVIDIA/garak/pull/1623
* Add résumé prompt injection examples by @Sid-V5 in https://github.com/NVIDIA/garak/pull/1616
* fix: binary inclusion of jailbreak components now selects correctly by @leondz in https://github.com/NVIDIA/garak/pull/1621
* Improve Jinja Injection Detection by @erickgalinkin in https://github.com/NVIDIA/garak/pull/1609

### Documentation
* docs: reorganise detector metrics out of code reference section by @leondz in https://github.com/NVIDIA/garak/pull/1617
* docs: improve docs on bar for acceptable probe; link these from other places by @leondz in https://github.com/NVIDIA/garak/pull/1647

### Tuning & fixes
* reporting: have `qual_review` support `report.jsonl` eval entries from 0.13.3 and earlier by @leondz in https://github.com/NVIDIA/garak/pull/1605
* linting, 2026 01 30 by @leondz in https://github.com/NVIDIA/garak/pull/1587
* bump black version to 26.x, remove pin by @leondz in https://github.com/NVIDIA/garak/pull/1591
* Fix core cfg listed twice in report in default run  by @arjun-krishna1 in https://github.com/NVIDIA/garak/pull/1544
* task: bump yarn dependencies to ensure secure posture by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1611
* tests: reorg test assets into one dir by @leondz in https://github.com/NVIDIA/garak/pull/1615
* task: update yarn based dependencies by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1631
* task: add report tests automation by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1632
* tests: force brew installed libmagic version to 5.46 by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1635
* task: ensure calls to json.dumps allow non-ascii for language support by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1630
* fix: detector description and CLI follow-ups by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1625
* chore: add optional pre-commit hook for garak-report by @otavionvidia in https://github.com/NVIDIA/garak/pull/1594
* lock litellm by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1648
* update `yarn.lock` for latest packages by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1661
* fix: use parameterized SQL queries in report_digest.py by @gn00295120 in https://github.com/NVIDIA/garak/pull/1645
* task: add preliminary build prep action by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1570

## New Contributors
* @Sid-V5 made their first contribution in https://github.com/NVIDIA/garak/pull/1616
* @dyrtyData made their first contribution in https://github.com/NVIDIA/garak/pull/1379
* @leoneperdigao made their first contribution in https://github.com/NVIDIA/garak/pull/1571
* @gn00295120 made their first contribution in https://github.com/NVIDIA/garak/pull/1645

**Full Changelog**: https://github.com/NVIDIA/garak/compare/v0.14.0...v0.14.1