v2.42.0

spider-rs/spiderv2.42.0Feb 1, 2026by j-mendez

AI Summary

Added full W3C WebDriver protocol support via thirtyfour crate for browser automation with support for Chrome, Firefox, Edge, and remote connections.

Key Highlights

  • Full W3C WebDriver protocol support via thirtyfour crate
  • Browser support: Chrome, Firefox, Edge
  • Remote connections: Selenium Grid, BrowserStack, SauceLabs
  • Stealth mode to hide automation markers
  • Automation marker cleanup for anti-detection

New Features

  • WebDriver support for browser automation
  • Multiple browser support (Chrome, Firefox, Edge)
  • Remote server connections (Selenium Grid, BrowserStack)
  • Stealth mode with navigator property overrides
  • CDP marker cleanup for stealth mode

Full Release Notes

## WebDriver Support

Full W3C WebDriver protocol support via `thirtyfour` crate for Selenium Grid, remote browsers, and cross-browser testing.

```rust
use spider::website::Website;
use spider::features::webdriver_common::{WebDriverConfig, WebDriverBrowser};

let mut website = Website::new("https://example.com")
    .with_webdriver(
        WebDriverConfig::new()
            .with_server_url("http://localhost:4444")
            .with_browser(WebDriverBrowser::Chrome)
            .with_headless(true)
    )
    .build()
    .unwrap();

website.crawl().await;
```

## Features

- **Browser Support**: Chrome, Firefox, Edge via WebDriver protocol
- **Remote Connections**: Selenium Grid, BrowserStack, SauceLabs compatible
- **Stealth Mode**: Automation marker cleanup when `webdriver_stealth` enabled
  - Hides `navigator.webdriver`
  - Spoofs `navigator.deviceMemory`
  - Cleans CDP markers (`cdc_`, `$cdc_`)
  - Removes Selenium markers
- **Automation Scripts**: Reuses existing chrome automation configuration

## Feature Flags

```toml
[dependencies]
spider = { version = "2.42", features = ["webdriver"] }
# With stealth:
spider = { version = "2.42", features = ["webdriver", "webdriver_stealth"] }
```

## New Files
- `spider/src/features/webdriver.rs`
- `spider/src/features/webdriver_common.rs`
- `spider/src/features/webdriver_args.rs`
- `examples/webdriver.rs`
- `examples/webdriver_remote.rs`