Starstruk Logo MarkStarstruk

v2.45.24

spider-rs/spiderv2.45.24Feb 21, 2026by github-actions[bot]

AI Summary

Major performance release introducing cache-first fast path (5-50ms vs 1-3s), deferred Chrome processing, work-stealing for hedged requests, and io_uring for high-throughput file I/O. Also adds per-round model pool routing for the agent and several bug fixes for Chrome mode and SSL handling.

Key Highlights

  • Cache-first fast path skips browser/HTTP when cache has data
  • Deferred Chrome processes multi-page crawls from cache before launching browser
  • Work-stealing (hedged requests) for parallel retry of slow crawl requests
  • io_uring StreamingWriter for high-throughput file I/O on Linux
  • Per-round model pool routing for agent

New Features

  • Cache-first fast path (~5-50ms vs 1-3s)
  • Deferred Chrome for multi-page crawls
  • Work-stealing (hedged requests)
  • io_uring StreamingWriter
  • Per-round model pool routing
  • Comprehensive router tests (30+ multi-LLM tests)
  • Chrome mode honors wait_for config
  • Smart mode lifecycle waiting
  • Auto-retry www. URLs on SSL errors
  • Reject empty HTML from all cache paths
  • Default no_sandbox() for headless Chrome
  • --wait-for CLI flag

Full Release Notes

## What's New

### Performance
- **Cache-first fast path** — skip browser/HTTP entirely when cache has data (~5-50ms vs 1-3s)
- **Deferred Chrome** — process multi-page crawls from cache before launching a browser
- **Work-stealing (hedged requests)** — parallel retry for slow crawl requests
- **io_uring** — StreamingWriter for high-throughput file I/O on Linux

### Agent
- **Per-round model pool routing** — route cheap rounds to fast models, complex rounds to capable ones
- **Comprehensive router tests** — 30+ multi-LLM reliability tests

### Fixes
- Chrome mode honors `wait_for` config (e.g. `idle_network0`) before HTML extraction
- Smart mode lifecycle waiting matches Chrome coverage without re-fetch
- Auto-retry www. URLs on SSL protocol errors
- Reject empty HTML from all cache paths
- Default `no_sandbox()` for headless Chrome (#354)

### CLI
- `--wait-for` flag for page load strategies (#352)

### Housekeeping
- Clippy warnings and formatting cleanup across workspace
- Rewritten README, CONTRIBUTING.md, CHANGELOG backfill
- Issue/PR templates, security policy, CI workflows (lint, audit, release)