v2.41.1
spider-rs/spiderv2.41.1Feb 1, 2026by j-mendez
AI Summary
Added WebDriver support via thirtyfour crate with automatic crawl integration, multiple browser support, and stealth mode capabilities.
Key Highlights
- Automatic crawl integration with webdriver_config
- Multiple browser support: Chrome, Firefox, Edge
- Remote server support: Selenium Grid, BrowserStack
- Stealth mode to reduce automation detection
- Feature flags for WebDriver configuration
New Features
- WebDriver integration via thirtyfour crate
- Automatic crawl integration when webdriver_config is set
- Multiple browser support with feature flags
- Remote server connections
- Stealth mode with navigator property overrides
Full Release Notes
# v2.41.0 - WebDriver Support
This release adds WebDriver support via the `thirtyfour` crate, enabling browser
automation using the W3C WebDriver protocol. Connect to remote Selenium Grid,
chromedriver, geckodriver, and more.
## New Features
### WebDriver Integration
- **Automatic crawl integration**: When `webdriver_config` is set, `crawl()`
automatically uses WebDriver instead of raw HTTP
- **Multiple browser support**: Chrome, Firefox, and Edge
- **Remote server support**: Connect to Selenium Grid, BrowserStack, or any W3C
WebDriver-compatible server
- **Stealth mode**: Navigator property overrides to reduce automation detection
### New Feature Flags
| Flag | Description |
|------|-------------|
| `webdriver` | Core WebDriver support via thirtyfour |
| `webdriver_headed` | Run browser in headed (visible) mode |
| `webdriver_stealth` | Enable stealth mode to hide automation |
| `webdriver_screenshot` | Screenshot capture support |
| `webdriver_chrome` | Chrome-specific features |
| `webdriver_firefox` | Firefox-specific features |
| `webdriver_edge` | Edge-specific features |
## Usage
```rust
use spider::website::Website;
use spider::features::webdriver_common::{WebDriverConfig, WebDriverBrowser};
let mut website = Website::new("https://example.com")
.with_webdriver(
WebDriverConfig::new()
.with_server_url("http://localhost:4444")
.with_browser(WebDriverBrowser::Chrome)
.with_headless(true)
)
.build()
.unwrap();
// crawl() automatically uses WebDriver
website.crawl().await;
Examples
Three new examples added:
- webdriver - Basic WebDriver crawling
- webdriver_remote - Remote Selenium Grid connection
- webdriver_screenshot - Screenshot capture
# Start chromedriver
chromedriver --port=4444
# Or use Selenium Grid with Docker
docker run -d -p 4444:4444 --shm-size="2g" selenium/standalone-chrome:latest
# Run example
cargo run --example webdriver --features="webdriver webdriver_stealth"
Full Changelog
https://github.com/spider-rs/spider/compare/v2.40.5...v2.41.1