Starstruk Logo MarkStarstruk

v0.1.8

feyninc/mandexv0.1.8Mar 23, 2026by chonknick

AI Summary

Overhauls reranking performance by switching to a single pass rerank, utilizing optimized ONNX Runtime (ORT) format, and enabling multi-threaded inference.

Key Highlights

  • Reranking performance overhaul resulting in 3.3x faster speeds
  • Single rerank pass across packages eliminates redundant ONNX session loads
  • Adoption of ORT format for faster model deserialization
  • Multi-threaded inference support using half available cores (capped at 4)

New Features

  • Single rerank pass
  • ORT format support
  • Multi-threaded inference

Full Release Notes

## What's Changed

### Reranking Performance Overhaul (3.3x faster)

- **Single rerank pass** — collect all FTS5 results across packages, rerank once instead of per-package (eliminates redundant ONNX session loads)
- **ORT format** — pre-optimized flatbuffer model (`reranker.ort`) for faster deserialization, with automatic fallback to `.onnx`
- **mmap model loading** — zero-copy memory-mapped file access via `commit_from_memory_directly`
- **Multi-threaded inference** — uses half available cores (capped at 4) for ONNX intra-op parallelism

### Benchmarks

| Scenario | v0.1.7 | v0.1.8 | Speedup |
|----------|--------|--------|---------|
| All packages (11) | 1.01s | **0.31s** | 3.3x |
| Single package | 0.09s | **0.04s** | 2.3x |