v0.1.7 v0.1.8
cheaper-inference/OmniRouteMar 23, 2026 → Mar 23, 20260 days between releases
What's New
Reranking speed increased 3.3x with single-pass strategy and ORT models.
New Features
- Single rerank pass across all packages instead of per-package reranking.
- ORT format model support with automatic fallback to .onnx.
- Multi-threaded inference capped at 4 cores for parallel processing.
- Zero-copy memory-mapped file access for faster model loading.
Improvements
- 3.3x faster reranking performance across all packages.
- Pre-optimized ORT format for faster deserialization.
- Zero-copy memory-mapped file access via mmap.
- Multi-threaded inference using half available cores.
Upgrade Notes
Ensure you have the new `reranker.ort` model file, though the system supports automatic fallback to `.onnx`. The reranking logic now uses a single pass across packages instead of per-package reranking.
The Two Releases
v0.1.7
Mar 23, 2026· 7 months ago
Replaced the HuggingFace tokenizers library with `tokie` to achieve massive performance improvements in reranking, reducing initialization time and dependencies while maintaining 100% accuracy.
Full release notesv0.1.8
Mar 23, 2026· 7 months ago
Overhauled reranking performance by 3.3x by implementing a single pass reranking strategy, using pre-optimized ORT models, and enabling zero-copy memory-mapped file access with multi-threaded inference.
Full release notes