v0.1.8

shader-effects-inc/shadersv0.1.8Mar 23, 2026by chonknick

Upgrading?

AI Summary

Performance overhaul for reranking using multi-threading, optimized ONNX formats, and memory-mapped model loading.

Key Highlights

  • 3.3x faster reranking performance
  • Single rerank pass instead of per-package
  • Mmap model loading for zero-copy access
  • Multi-threaded inference

New Features

  • Multi-threaded inference
  • Mmap model loading
  • Optimized reranking performance

Full Release Notes

## What's Changed

### Reranking Performance Overhaul (3.3x faster)

- **Single rerank pass** — collect all FTS5 results across packages, rerank once instead of per-package (eliminates redundant ONNX session loads)
- **ORT format** — pre-optimized flatbuffer model (`reranker.ort`) for faster deserialization, with automatic fallback to `.onnx`
- **mmap model loading** — zero-copy memory-mapped file access via `commit_from_memory_directly`
- **Multi-threaded inference** — uses half available cores (capped at 4) for ONNX intra-op parallelism

### Benchmarks

| Scenario | v0.1.7 | v0.1.8 | Speedup |
|----------|--------|--------|---------|
| All packages (11) | 1.01s | **0.31s** | 3.3x |
| Single package | 0.09s | **0.04s** | 2.3x |

Related Repositories