Starstruk Logo MarkStarstruk

v1.99.5

spider-rs/spiderv1.99.5Jul 13, 2024by j-mendez

AI Summary

Adds support for custom JavaScript execution scripts to handle specific scenarios like login pages, alongside performance optimizations.

Key Highlights

  • New `website.with_execution_scripts` method for custom JS
  • Fix Chrome wait_for selector and event handling
  • Major perf improvement for regex feature and robots.txt handling

New Features

  • New `website.with_execution_scripts` feature allowing hashmap of scripts

Full Release Notes

# Whats Changed

Use `website.with_execution_scripts` with a hashmap of exact paths and JS to run on the page before returning the result.
This can help writing one off code to unblock crawling a path like a login page.

* Fix chrome wait_for selector and event handling 
* Major perf improvement when using the `regex` feature flag and robots.txt handling.

**Full Changelog**: https://github.com/spider-rs/spider/compare/v1.98.8...v1.99.5