v0.1.7

Netw0rkNoob/VulnClawv0.1.7Mar 23, 2026by chonknick

AI Summary

Replaced the HuggingFace tokenizer with a faster, lightweight alternative to improve initialization speed.

Key Highlights

~20-50x faster tokenizer initialization.
~260 fewer transitive dependencies.
100% token-accurate results.
No runtime dependency on HuggingFace Hub.

New Features

Replaced HuggingFace `tokenizers` with `tokie`
Tokenizer served from CDN (cdn.mandex.dev/models/tokenizer.tkz)

Full Release Notes

## What's Changed

- Replace HuggingFace `tokenizers` with `tokie` for reranking
  - ~20-50x faster tokenizer initialization (pre-built .tkz from CDN vs JSON parsing)
  - ~260 fewer transitive dependencies
  - 100% token-accurate, identical reranking results
  - No runtime dependency on HuggingFace Hub
- Tokenizer served from `cdn.mandex.dev/models/tokenizer.tkz` alongside reranker model