Starstruk Logo MarkStarstruk

v2.47.75

NVIDIA/personaplexv2.47.75Mar 20, 2026by j-mendez

AI Summary

Significant performance improvements via SIMD, zero-copy parsing, and architectural updates like PageData/Crawler trait abstractions, alongside proxy support for LLM requests and Chrome remote cache features.

Key Highlights

  • PageData & Crawler trait abstractions
  • Proxy support for LLM HTTP requests
  • Chrome remote_addr via CDP
  • Remote cache for Chrome responses
  • SIMD-accelerated byte scanning

New Features

  • LLM HTTP request proxy support
  • Chrome remote_addr via CDP
  • Remote cache for Chrome responses
  • Trait abstractions for extensible crawl pipelines

Full Release Notes

## What's New

- **PageData & Crawler trait abstractions** for extensible crawl pipelines
- **Proxy support for LLM HTTP requests** (#378)
- **Chrome remote_addr** via CDP `Network.responseReceived`
- **Remote cache for Chrome responses** — dump & fallback support

## Performance

- SIMD-accelerated byte scanning (memchr), unrolled FNV hash
- Trie: `Box<str>` keys + manual byte-walk + memchr dot scan
- Bloom filter bitmask addressing + inline early-exit
- Zero-alloc DNS cache hits via `Arc<[SocketAddr]>`
- Skip robots.txt for single-page crawls, TCP keepalive always
- Batch LRU cache eviction to reduce write-lock hold time
- `cache_mem` auto-enables `skip_browser` for Chrome mode
- Remove unnecessary Box allocations and redundant string conversions

## Fixes

- Fall back to HTTP crawl when Chrome is unavailable (#373)
- Skip redirect-caused `net::ERR_ABORTED` instead of aborting navigation
- Deterministic CDP event listener shutdown via watch channel
- Remote cache fallback + 3s timeout for chrome_remote_cache path
- Add decentralized stubs for `set_url_parsed_direct` and `links_full`

## Deps

- strum 0.28, phf 0.13, gemini-rust 1.7, chromey 2, llm_models_spider 0.1

**Full Changelog**: https://github.com/spider-rs/spider/compare/v2.47.52...v2.47.75