spider_agent-v0.4.0

spider-rs/spiderspider_agent-v0.4.0Feb 3, 2026by j-mendez

AI Summary

Major performance release for spider_agent with SmartCache, ChainExecutor, BatchExecutor, PrefetchManager, and ModelRouter enabling intelligent automation workflows with parallel execution and predictive loading.

Key Highlights

SmartCache with size-aware LRU and automatic cleanup
ChainExecutor with parallel step execution and dependency analysis
BatchExecutor for efficient batch processing
PrefetchManager for predictive page loading
ModelRouter with complexity scoring and cost-tier routing

New Features

SmartCache: Size-aware LRU cache with TTL, automatic cleanup on memory pressure, statistics tracking
ChainExecutor: Parallel step execution, response caching with TTL, configurable concurrency limits, step timeout support
BatchExecutor: Process multiple items with configurable batch sizes, parallel execution within batches
PrefetchManager: Background URL prefetching, automatic cache management, concurrent prefetch limits
ModelRouter: Task complexity scoring, user-configurable model policies, cost tier constraints (Low/Medium/High), latency-aware routing
MessageContent helper methods: as_text(), full_text(), is_text(), has_images()
Default ModelPolicy now allows High tier routing

Full Release Notes

## Spider Agent v0.4.0

### Performance Optimizations

This release adds several performance optimizations for automation workflows:

#### Smart Caching
- **SmartCache**: Size-aware LRU cache with automatic cleanup
  - Bounded memory usage with configurable limits
  - TTL-based expiration
  - Automatic cleanup on memory pressure
  - Statistics tracking (hits, misses, evictions)

#### High-Performance Execution
- **ChainExecutor**: Parallel step execution for automation chains
  - Analyzes dependencies for optimal parallelization
  - Response caching with TTL
  - Configurable concurrency limits
  - Step timeout support

- **BatchExecutor**: Efficient batch processing
  - Process multiple items with configurable batch sizes
  - Parallel execution within batches
  - Index-aware processing option

- **PrefetchManager**: Predictive page loading
  - Prefetch URLs in the background
  - Automatic cache management
  - Concurrent prefetch limits

#### Smart Model Routing
- **ModelRouter**: Intelligent model selection based on task complexity
  - Task analysis for complexity scoring
  - User-configurable model policies
  - Cost tier constraints (Low/Medium/High)
  - Latency-aware routing

### Other Changes
- Added `MessageContent` helper methods: `as_text()`, `full_text()`, `is_text()`, `has_images()`
- Default `ModelPolicy` now allows High tier routing
- Fixed compilation warnings

### Full Changelog
https://github.com/spider-rs/spider/compare/spider_agent-v0.3.0...spider_agent-v0.4.0