Starstruk Logo MarkStarstruk

v0.2.93

D4Vinci/Scraplingv0.2.93Jan 31, 2025by D4Vinci

AI Summary

A major refactor focusing on parser engine consistency, significantly boosting speed, and enhancing the TextHandler class to support better method chaining.

Key Highlights

  • Consistent return types across all parser engine methods
  • Major speed increases between 5-60% for searching methods
  • Added `below_elements` property to DOM tree traversal
  • TextHandler slicing, indexing, and split methods return `TextHandler`

New Features

  • Refactored Adaptor class for cleaner and faster code
  • StealthyFetcher can now click Captchas in iframes like Cloudflare Turnstile
  • Improved auto-completion and type hints for Adaptor, TextHandler, and TextHandlers

Full Release Notes

**This is an essential update for everyone to fully enjoy Scrapling as it's intended**

## What's changed
1. The return type is now consistent across all the parser engine so you will always get a return type as one of these `Adaptor`, `Adaptors`, `TextHandler`, `TextHandlers`, `None`, and a list in case you have mixed results like combined CSS selector. This allows a better coding experience with minimum manual type checking, makes the library more stable, and makes chaining methods always possible.
2. Most of the parser engine especially the `Adaptor` class got refactored to a cleaner version and most importantly a faster version. So now almost all the methods/properties, especially the searching methods, got a speed increase between 5-40%. Some methods got bigger speed boosts like `find_by_regex` got a ~60% speed boost! The automatch feature got a small ~5% speed boost.
3. Fixed logic bugs with the `find_all`/`find` methods that made the passed filters used in OR fashion and other times as an AND. So now all elements returned need to fulfill all filters you pass.
4. Now all regex-related methods return `TextHandler`/`TextHandlers` for easier methods chaining.
5. **Added a new** `below_elements` property that returns an `Adaptors` object of all elements under the current element in the DOM tree.
6. Now all methods/properties that were returning HTML source as string are now returning it as `TextHandler` so you can do regex easily on it etc...
7. StealthyFetcher is now a bit faster and more stealthy. Also, now it's possible to click Captchas in iframes like **Cloudflare Turnstile**.
8. The auto-completion and type hints improved a lot in nearly half the library. Especially `Adaptor`, `TextHandler`, and `TextHandlers`.
9. Now slicing `TextHandler`, accessing by index, or using the `split` method returns another `TextHandler` instead of the standard Python string. Now almost all standard string operations/methods return other `Texthandler` instead of standard string to make chaining methods/functions always possible.
10. Fixed some small bugs and typos. For example, the Fetcher async_put was doing post request instead of put request 😶‍🌫️ 
11. Improved the README a bit till I finish the documentation website.

This was supposed to be a small update till version 0.3 but thought to make it better.

Thanks for all your support!

---

## Shoutout to our biggest Sponsor: [Scrapeless](https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci)
<a href="https://www.scrapeless.com/?utm_source=github&utm_medium=ads&utm_campaign=scraping&utm_term=D4Vinci"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/scrapeless.jpg" height="500" alt="Scrapeless Banner" ></a>