v0.2.91

D4Vinci/Scraplingv0.2.91Dec 19, 2024by D4Vinci

AI Summary

Fixes Playwright default behavior issues that caused incorrect status codes on redirect-heavy pages and introduces SOCKS proxy support.

Key Highlights

Fixed logging to show on every request instead of just the first
Fixed Playwright returning initial 403 status on redirect pages
Added support for SOCKS proxies in the `Fetcher` class

Full Release Notes

# What's changed
- Fixed a bug where the logging fetch logging sentence was showing in the first request only.
- The default behavior for Playwright API while browsing a page is returning the first response that fulfills the load state given to the `goto` method `["load", "domcontentloaded", "networkidle"]` so if a website has a wait page like Cloudflare's one that redirects you to the real website afterward, Playwright will return the first status code which in this case would be something like 403. This update solves this issue for both `PlaywrightFetcher` and `StealthyFetcher` as both are using Playwright API so the result depends on Playwright's default behavior no more.
- Added support for proxies that use SOCKS proxies in the `Fetcher` class.
- Fixed the type hint for the `wait_selector_state` argument so now it will show the accurate values you should use while auto-completing.

---

> [!NOTE] 
> A friendly reminder that maintaining and improving `Scrapling` takes a lot of time and effort which I have been happily doing for months even though it's becoming harder. So, if you like `Scrapling` and want it to keep improving, you can help by supporting me through the [Sponsor button](https://github.com/sponsors/D4Vinci).