spider Releases
223 releases of spider-rs/spider
- v2.7.1
Enhances Chrome remote connection management, proxy support, and retry mechanisms.
Sep 30, 2024
- v2.6.15
Fixes link parsing for redirects and adds configuration options for TLS and host headers.
Sep 22, 2024
- v2.6.2HTML Transformations
Open-sourced transformation utilities for high-performance content output conversion (Markdown, Text).
Sep 21, 2024
- v2.5.3
Optimized visited link storage and lookup using string interning to reduce memory usage.
Sep 14, 2024
- v2.4.1
Significant performance improvements for screenshot functionality using Chrome parameters.
Sep 9, 2024
- v2.3.5
Major performance improvements for Chrome by enabling concurrent request interception.
Sep 8, 2024
- v2.2.18
Enabled automatic locale detection with proper HTML encoding support.
Aug 29, 2024
- v2.1.9
Bug fixes for Chrome hanging issues and new builder method to return page links.
Aug 26, 2024
- v2.0.6
Added HTTP response cookies map and fixed Chrome file system feature flag build.
Aug 20, 2024
- v1.99.30
Introduced web automation steps for Chrome, including viewport handling and specific actions.
Aug 7, 2024
- v1.99.21
Added ad-blocking capabilities over the network using Chrome interception.
Aug 7, 2024
- v1.99.18
Fixed Chrome file system storing functionality.
Aug 5, 2024
- v1.99.16
Addresses configuration stability and refines browser emulation while improving performance handling of assets in smart mode.
Aug 1, 2024
- v1.99.10
Major performance increase for CLI scraping is introduced alongside a new utility crate designed for manual content extraction using CSS selectors.
Jul 19, 2024
- v1.99.5
Adds support for custom JavaScript execution scripts to handle specific scenarios like login pages, alongside performance optimizations.
Jul 13, 2024
- v1.98.8
Improves robots.txt parsing reliability for root URLs and adds serialization support for wait states.
Jul 4, 2024
- v1.98.4
Users can now configure the maximum resource size limit for downloads via an environment variable.
Jul 2, 2024
- v1.98.3
Adds URL whitelisting functionality to restrict crawling to specific paths and fixes issues with sitemap parsing.
Jun 19, 2024
- v1.2.0Open-Sora v1.2.0 Released Today!
Released Open-Sora v1.2, introducing a video compression network, rectified-flow training, and better evaluation metrics.
Jun 18, 2024
- v1.97.14
Fixes Chrome User-Agent spoofing issues and introduces a shared queue strategy for fair workload distribution.
Jun 13, 2024