v0.1.7

Gen-Verse/MMaDAv0.1.7Mar 23, 2026by chonknick

AI Summary

Replaced HuggingFace tokenizers with a custom `tokie` library to achieve significant performance gains and reduce dependencies.

Key Highlights

~20-50x faster tokenizer initialization
~260 fewer transitive dependencies
100% token-accurate reranking results
No runtime dependency on HuggingFace Hub

New Features

Custom tokenizer library (`tokie`)
Tokenizer served from CDN

Full Release Notes

## What's Changed

- Replace HuggingFace `tokenizers` with `tokie` for reranking
  - ~20-50x faster tokenizer initialization (pre-built .tkz from CDN vs JSON parsing)
  - ~260 fewer transitive dependencies
  - 100% token-accurate, identical reranking results
  - No runtime dependency on HuggingFace Hub
- Tokenizer served from `cdn.mandex.dev/models/tokenizer.tkz` alongside reranker model