v0.1.8
kazuto-07/TTS-cacherv0.1.8Mar 23, 2026by chonknick
Upgrading?
AI Summary
Significant performance improvements to the reranking system, reducing execution time by up to 3.3x through architectural changes.
Key Highlights
- Single rerank pass eliminates redundant ONNX session loads
- Use of ORT format for optimized flatbuffer model deserialization
- Memory-mapped file access for zero-copy model loading
- Multi-threaded inference using available cores
New Features
- Single rerank pass across packages
- ORT format model support with automatic fallback
- Memory-mapped model loading
- Multi-threaded ONNX inference
Full Release Notes
## What's Changed ### Reranking Performance Overhaul (3.3x faster) - **Single rerank pass** — collect all FTS5 results across packages, rerank once instead of per-package (eliminates redundant ONNX session loads) - **ORT format** — pre-optimized flatbuffer model (`reranker.ort`) for faster deserialization, with automatic fallback to `.onnx` - **mmap model loading** — zero-copy memory-mapped file access via `commit_from_memory_directly` - **Multi-threaded inference** — uses half available cores (capped at 4) for ONNX intra-op parallelism ### Benchmarks | Scenario | v0.1.7 | v0.1.8 | Speedup | |----------|--------|--------|---------| | All packages (11) | 1.01s | **0.31s** | 3.3x | | Single package | 0.09s | **0.04s** | 2.3x |
Related Repositories
public-apis/public-apis★ 491,268
A collective list of free APIs
NousResearch/hermes-agent★ 256,134
The agent that grows with you
yt-dlp/yt-dlp★ 200,318
A feature-rich command-line audio/video downloader
microsoft/markitdown★ 191,598
Python tool for converting files and office documents to Markdown.
Significant-Gravitas/AutoGPT★ 187,705
AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.