v2.43.0

spider-rs/spiderv2.43.0Feb 1, 2026by j-mendez

AI Summary

Added token usage tracking and extraction support to RemoteMultimodalEngine with AutomationUsage struct and custom extraction capabilities.

Key Highlights

  • AutomationUsage struct for tracking prompt, completion, and total tokens
  • extra_ai_data option for extraction mode
  • extraction_prompt for custom extraction instructions
  • screenshot capture for final page state

New Features

  • Token usage tracking conforming to OpenAI API format
  • Extraction capabilities with custom prompts
  • Screenshot capture support
  • Automated data storage on Page.extra_remote_multimodal_data

Full Release Notes

## What's New

### Token Usage Tracking for RemoteMultimodalEngine

The remote multimodal automation engine now tracks and returns token usage conforming to the OpenAI API format:

- `AutomationUsage` struct with `prompt_tokens`, `completion_tokens`, `total_tokens`
- Usage is accumulated across all inference rounds
- Stored on `Page.remote_multimodal_usage`

### Extraction Support

New extraction capabilities for RemoteMultimodalEngine, similar to the OpenAI integration:

- `extra_ai_data` - Enable extraction mode
- `extraction_prompt` - Custom extraction instructions
- `screenshot` - Capture final screenshot

Extracted data is automatically stored on `Page.extra_remote_multimodal_data` as `AutomationResults`.

### Example Usage

```rust
use spider::features::automation::RemoteMultimodalConfigs;

let mm = RemoteMultimodalConfigs::new(
    "http://localhost:11434/v1/chat/completions",
    "qwen2.5-vl",
)
.with_extra_ai_data(true)
.with_extraction_prompt(Some("Extract all product names and prices"))
.with_screenshot(true);

website.configuration.remote_multimodal = Some(Box::new(mm));

// After crawling, access on page:
for page in website.get_pages().await {
    if let Some(usage) = &page.remote_multimodal_usage {
        println!("Tokens: {:?}", usage);
    }
    if let Some(data) = &page.extra_remote_multimodal_data {
        for result in data {
            println!("Extracted: {:?}", result.content_output);
        }
    }
}
```

**Full Changelog**: https://github.com/spider-rs/spider/compare/v2.42.0...v2.43.0