Starstruk Logo MarkStarstruk

v0.9.0.16

NVIDIA/garakv0.9.0.16Sep 6, 2024by leondz

AI Summary

This release introduces new probes for hallucinations in Ruby, NPM, and Rust crates, along with a Groq API generator. It also includes refactoring for the generations parameter, secure HuggingFace assets, and tokenizer customization.

Key Highlights

  • New Groq API generator support
  • New hallucination probes for Ruby, NPM, and Rust crates
  • Latent prompt injection probe added
  • Refactor of generations parameter to be set on probes
  • Secure HuggingFace assets implementation

Breaking Changes

  • Renamed `knownbadsignatures` to `av_spam_scanning` (configuration change)

New Features

  • Groq API generator
  • Latent prompt injection probe
  • Ruby/NPM/Rust crate hallucination probes
  • Tokenizer customization in HFDetector
  • Z-score reporting in CLI

Full Release Notes

## What's Changed

### New plugins

* probe: topic pushing by @leondz in https://github.com/leondz/garak/pull/764
* probe: ruby package hallucination by @arjun-krishna1 in https://github.com/leondz/garak/pull/851
* probe: Latent prompt injection by @leondz in https://github.com/leondz/garak/pull/877
* probe: npm package hallucination by @arjun-krishna1 in https://github.com/leondz/garak/pull/861
* probe: rust crate hallucination by @arjun-krishna1 in https://github.com/leondz/garak/pull/873
* generator: Groq API by @mmilenkovic-groq in https://github.com/leondz/garak/pull/896

### New features

* extract `generations` param to be set on probes instead of generators by @jmartin-tech in https://github.com/leondz/garak/pull/837
* secure garak HF assets by @leondz in https://github.com/leondz/garak/pull/854
* Rename `knownbadsignatures` to `av_spam_scanning` by @leondz in https://github.com/leondz/garak/pull/850
* enable tokenizer customization in HFDetector by @jmartin-tech in https://github.com/leondz/garak/pull/855
* refactor calibration / z-score code, so z-scores can be shown in CLI at run time by @leondz in https://github.com/leondz/garak/pull/847

### Tuning & fixes

* add automation action for plugin_cache.json by @jmartin-tech in https://github.com/leondz/garak/pull/819
* add automation action for plugin_cache.json by @jmartin-tech in https://github.com/leondz/garak/pull/819
* Docs/readmefaqdc32 by @leondz in https://github.com/leondz/garak/pull/828
* fix initialization bug for kwargs in function by @soumilinandi in https://github.com/leondz/garak/pull/827
* suppress LiteLLM logging during import by @jmartin-tech in https://github.com/leondz/garak/pull/834
* relax `--probe_tags` as supported when other probe options are passed by @jmartin-tech in https://github.com/leondz/garak/pull/836
* rename harness test module by @leondz in https://github.com/leondz/garak/pull/848
* Surface underlying exception msg at plugin load by @leondz in https://github.com/leondz/garak/pull/846
* relax litellm provider constraint by @arjun-krishna1 in https://github.com/leondz/garak/pull/820
* plugin metadata tests by @leondz in https://github.com/leondz/garak/pull/849
* Add explanation of how we treat the word "vulnerability" by @leondz in https://github.com/leondz/garak/pull/852
* add faq items by @leondz in https://github.com/leondz/garak/pull/857
* test buff transform() methods, to get them to load and run by @leondz in https://github.com/leondz/garak/pull/856
* expand doc string that fails test requirements for verbosity by @jmartin-tech in https://github.com/leondz/garak/pull/859
* refresh tutorial on contributing a generator by @leondz in https://github.com/leondz/garak/pull/858
* avoid lambda in `_config` module level code by @jmartin-tech in https://github.com/leondz/garak/pull/860
* add logging dir, new generator questions to FAQ by @leondz in https://github.com/leondz/garak/pull/866
* revise default generations value to 5 by @leondz in https://github.com/leondz/garak/pull/868
* validate detector doc_uri by @leondz in https://github.com/leondz/garak/pull/865
* fix output error when parallel_requests > 1 and supports_multiple_generations is False by @arjun-krishna1 in https://github.com/leondz/garak/pull/864
* improved messaging & error handling around NIM names by @leondz in https://github.com/leondz/garak/pull/867
* look at the right var for NIM name checking by @leondz in https://github.com/leondz/garak/pull/871
* filter detector template in tests by @jmartin-tech in https://github.com/leondz/garak/pull/874
* have CLI UI render 100% w/o using scientific notation by @leondz in https://github.com/leondz/garak/pull/882
* Generator docs are now tested for as intended by @leondz in https://github.com/leondz/garak/pull/897
* reporting tweaks by @leondz in https://github.com/leondz/garak/pull/901

## New Contributors
* @soumilinandi made their first contribution in https://github.com/leondz/garak/pull/827
* @arjun-krishna1 made their first contribution in https://github.com/leondz/garak/pull/820
* @mmilenkovic-groq made their first contribution in https://github.com/leondz/garak/pull/896
* @greshake made their first contribution in https://github.com/leondz/garak/pull/877

**Full Changelog**: https://github.com/leondz/garak/compare/v0.9.0.15...v0.9.0.16