v2.36.123
spider-rs/spiderv2.36.123Apr 18, 2025by j-mendez
AI Summary
Fixes HTTP and smart mode request Host headers to prevent issues with redirects and fixes OpenAI automation usage. Also introduces antibot detection and improves sitemap handling.
Key Highlights
- Fix for Host header preventing proper redirects
- New antibot detection feature
- Fixes for sitemap XML handling
- Viewport browser handling fixes
New Features
- Antibot detection
- Sitemap whitelisting
- Viewport browser handling
Full Release Notes
# Whats Changed Major fix for http or smart mode request adding the Host header preventing proper redirects. Fix openai automation usage. * chore(website): fix client host header * chore(chrome,sitemap): fix sitemap handling xml * feat(antibot): add antibot detection * chore(chrome): fix viewport browser handling pages * chore(chrome): fix fingerprint execution script * chore(sitemap): add auto sitemap adding whitelisting **Full Changelog**: https://github.com/spider-rs/spider/compare/v2.36.67...v2.36.123