Starstruk Logo MarkStarstruk

v0.14.0

andrewyng/aisuitev0.14.0Feb 4, 2026by jmartin-tech

AI Summary

This release focuses on significant updates to the reporting system, including a complete HTML report redesign and modifications to JSONL output formats, alongside dependency updates and performance improvements.

Key Highlights

  • Redesigned HTML reports
  • Modified format of `eval` and `digest` entries in JSONL report
  • Removal of `--generate_autodan` CLI option
  • Updated OpenAI library to 2.x
  • Added support for JSON configuration

Breaking Changes

  • Modified format of `eval` and `digest` entries in JSONL report
  • Fully redesigned HTML reports
  • Removal of `--generate_autodan` CLI option

New Features

  • Support for JSON config as well as YAML
  • Expanded items in report `entry_type:eval`
  • Reporting tier biased security aggregate
  • Report Generator refactor
  • Added detector evaluation doc and benchmark results
  • Calibration data updates winter 2026

Full Release Notes

## What's Changed

### New features
* feature: support JSON config as well as YAML by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1490
* feature: remove `--generate_autodan` CLI option by @sinhaabhiraj0 in https://github.com/NVIDIA/garak/pull/1521
* expand items in report `entry_type:eval` by @leondz in https://github.com/NVIDIA/garak/pull/1547
* reporting: tier biased security aggregate by @leondz in https://github.com/NVIDIA/garak/pull/1329
* feature: Report Generator refactor by @otavionvidia in https://github.com/NVIDIA/garak/pull/1573
* feature: Add detector evaluation doc and benchmark results by @patriciapampanelli in https://github.com/NVIDIA/garak/pull/1597
* task: calibration data updates winter 2026 by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1599

#### Notice: Breaking changes in this release
* Modified format of `eval` and `digest` entires in JSONL report
  See "Run analysis" documentation for more details of compatibility expectations.
* Fully redesigned HTML reports
* Removal of `--generate_autodan` cli option

### Improved plugins
* Fix atkgen verbose output displaying incorrect conversation turns by @snehalvartak in https://github.com/NVIDIA/garak/pull/1542
* generator: relax constraint to specify valid openai model name by @leondz in https://github.com/NVIDIA/garak/pull/1557
* generators: check message count requested matches message count generated by @leondz in https://github.com/NVIDIA/garak/pull/1558
* fix: JavaScript detector regex improvement by @erickgalinkin in https://github.com/NVIDIA/garak/pull/1588

### Documentation
* docs: add guardion ai / promptintel to project by @leondz in https://github.com/NVIDIA/garak/pull/1530
* docs: add brief stubs for garak.analyze content by @leondz in https://github.com/NVIDIA/garak/pull/1569

### Tuning & fixes
* Make GoogleTranslator multiprocessing-safe by @motlaharsh0909-lgtm in https://github.com/NVIDIA/garak/pull/1543
* Fix: improved unsafe pickle strategy by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1545
* track which detector is reporting invalid pkg entry; only report once by @leondz in https://github.com/NVIDIA/garak/pull/1556
* arch: Remove many builtin configs by @leondz in https://github.com/NVIDIA/garak/pull/1555
* update: rm deprecated `gpt-3.5` model from defaults by @leondz in https://github.com/NVIDIA/garak/pull/1552
* task: update openai library to 2.x by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1550
* test: isolate _config in all tests by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1549
* update LangChain generator for 1.x library support by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1551
* add license to docs footer by @leondz in https://github.com/NVIDIA/garak/pull/1572
* guard for `None` content as response from target by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1574
* fix: 100% pass rate should not give 'elevated risk' for relative scores by @leondz in https://github.com/NVIDIA/garak/pull/1579
* Move generation output cardinality check into `Generator.generate()` by @leondz in https://github.com/NVIDIA/garak/pull/1578
* calling `as_dict()` on default instantiated `Attempt` now doesn't raise `TypeError` by @leondz in https://github.com/NVIDIA/garak/pull/1590
* task: remove fschat by @erickgalinkin in https://github.com/NVIDIA/garak/pull/1567
* task: report test tweaks and dependency lock updates by @otavionvidia in https://github.com/NVIDIA/garak/pull/1593
* Task: transformers 5 support by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1582
* task: exclude files not needed in distributed package by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1601
* reporting: factor up defcon calculation by @leondz in https://github.com/NVIDIA/garak/pull/1581
* rm `jinja2` direct dependency requirement by @leondz in https://github.com/NVIDIA/garak/pull/1596
* task: update huggingface config pattern by @jmartin-tech in https://github.com/NVIDIA/garak/pull/1602
* reporting: stabilise tbsa hashes by @leondz in https://github.com/NVIDIA/garak/pull/1603


## New Contributors
* @snehalvartak made their first contribution in https://github.com/NVIDIA/garak/pull/1542
* @sinhaabhiraj0 made their first contribution in https://github.com/NVIDIA/garak/pull/1521
* @motlaharsh0909-lgtm made their first contribution in https://github.com/NVIDIA/garak/pull/1543
* @otavionvidia made their first contribution in https://github.com/NVIDIA/garak/pull/1573

**Full Changelog**: https://github.com/NVIDIA/garak/compare/v0.13.3...v0.14.0