Starstruk Logo MarkStarstruk

v0.1.8

asheshgoplani/agent-deckv0.1.8Mar 23, 2026by chonknick

AI Summary

Overhauled reranking performance with optimizations including single-pass processing, ORT format support, and multi-threading.

Key Highlights

  • Single rerank pass eliminating redundant ONNX session loads
  • ORT format for faster model deserialization
  • Mmap model loading for zero-copy memory access
  • Multi-threaded inference (up to 4 cores)

New Features

  • ORT format support
  • Mmap model loading
  • Multi-threaded inference

Full Release Notes

## What's Changed

### Reranking Performance Overhaul (3.3x faster)

- **Single rerank pass** — collect all FTS5 results across packages, rerank once instead of per-package (eliminates redundant ONNX session loads)
- **ORT format** — pre-optimized flatbuffer model (`reranker.ort`) for faster deserialization, with automatic fallback to `.onnx`
- **mmap model loading** — zero-copy memory-mapped file access via `commit_from_memory_directly`
- **Multi-threaded inference** — uses half available cores (capped at 4) for ONNX intra-op parallelism

### Benchmarks

| Scenario | v0.1.7 | v0.1.8 | Speedup |
|----------|--------|--------|---------|
| All packages (11) | 1.01s | **0.31s** | 3.3x |
| Single package | 0.09s | **0.04s** | 2.3x |