v0.1.8

Gen-Verse/MMaDAv0.1.8Mar 23, 2026by chonknick

AI Summary

Optimized reranking performance through multi-threading, memory mapping, and optimized model formats, resulting in a 3.3x speedup.

Key Highlights

3.3x faster reranking for all packages
Single rerank pass instead of per-package (eliminates redundant ONNX loads)
ORT format (flatbuffer) with automatic fallback to `.onnx`
mmap model loading for zero-copy memory access
Multi-threaded inference using half available cores (capped at 4)

New Features

Multi-threaded inference
ORT format support
Memory-mapped model loading

Full Release Notes

## What's Changed

### Reranking Performance Overhaul (3.3x faster)

- **Single rerank pass** — collect all FTS5 results across packages, rerank once instead of per-package (eliminates redundant ONNX session loads)
- **ORT format** — pre-optimized flatbuffer model (`reranker.ort`) for faster deserialization, with automatic fallback to `.onnx`
- **mmap model loading** — zero-copy memory-mapped file access via `commit_from_memory_directly`
- **Multi-threaded inference** — uses half available cores (capped at 4) for ONNX intra-op parallelism

### Benchmarks

| Scenario | v0.1.7 | v0.1.8 | Speedup |
|----------|--------|--------|---------|
| All packages (11) | 1.01s | **0.31s** | 3.3x |
| Single package | 0.09s | **0.04s** | 2.3x |