Starstruk Logo MarkStarstruk

v3.18.0

VoiceBlender/voiceblenderv3.18.0Aug 4, 2026by apify-service-account

AI Summary

This release focuses on stabilizing core crawling functionality with numerous bug fixes, particularly around memory storage, request handling, and sitemap parsing. It also introduces security improvements and enhancements to Playwright and Puppeteer integrations.

Key Highlights

  • Fixed `enqueueLinksByClickingElements` to accept any `clickOptions`.
  • Improved memory storage security by preventing storage name escaping.
  • Enhanced sitemap handling to recognize `sitemap_index` URLs.
  • Added security fixes to prevent ReDoS in social email regex.
  • Updated Playwright and Puppeteer to only intercept non-GET requests.

New Features

  • Infer default-route handler userData from its schema in the core crawler.
  • Honor redirect handler for session cookie parity in the impit-client.

Full Release Notes

# [3.18.0](https://github.com/apify/crawlee/compare/v3.17.0...v3.18.0) (2026-08-04)

### Bug Fixes

* allow any `clickOptions` in `enqueueLinksByClickingElements` ([#3823](https://github.com/apify/crawlee/issues/3823)) ([abcbfb1](https://github.com/apify/crawlee/commit/abcbfb158ab3edbb9e7e4c5c6f1d937dea9bb4dd))
* avoid duplicate final crawler persistence ([#3866](https://github.com/apify/crawlee/issues/3866)) ([79dd6c4](https://github.com/apify/crawlee/commit/79dd6c41eff144914ddb7fa6e04a65e9b8b762e0)), closes [#3167](https://github.com/apify/crawlee/issues/3167)
* **basic-crawler:** declare `@apify/datastructures` as a dependency ([#3815](https://github.com/apify/crawlee/issues/3815)) ([4e9d227](https://github.com/apify/crawlee/commit/4e9d227d7711f10662800d4079c4f958368bc063))
* **core:** avoid re-submitting already-enqueued requests in addRequestsBatched ([#3843](https://github.com/apify/crawlee/issues/3843)) ([704c269](https://github.com/apify/crawlee/commit/704c269806503d36ad2c629a63f3a4944109930c)), closes [#2456](https://github.com/apify/crawlee/issues/2456) [#2456](https://github.com/apify/crawlee/issues/2456) [#2456](https://github.com/apify/crawlee/issues/2456) [#3120](https://github.com/apify/crawlee/issues/3120)
* **core:** cap retries for unprocessed requests in addRequestsBatched ([#3765](https://github.com/apify/crawlee/issues/3765)) ([b3170a6](https://github.com/apify/crawlee/commit/b3170a60c5c0dbb53f4078555df7551cd0b77967)), closes [#3764](https://github.com/apify/crawlee/issues/3764)
* **core:** clear `AutoscaledPool.pause()` polling interval on timeout ([#3847](https://github.com/apify/crawlee/issues/3847)) ([42b8914](https://github.com/apify/crawlee/commit/42b89146a372b639eaaf50fbda9ff16b026b3344))
* **core:** ignore explicitly undefined baseUrl in enqueueLinks helpers ([#3924](https://github.com/apify/crawlee/issues/3924)) ([a58a9c1](https://github.com/apify/crawlee/commit/a58a9c1417c34c2a45a10f3855c4bee721a480ae))
* **core:** keep crawler enqueue limits when options carry explicit undefined ([#3927](https://github.com/apify/crawlee/issues/3927)) ([5f1b0ab](https://github.com/apify/crawlee/commit/5f1b0ab480f297100d2db70f646825b92f70f21a)), closes [#3924](https://github.com/apify/crawlee/issues/3924) [#3924](https://github.com/apify/crawlee/issues/3924)
* **core:** keep same-domain filtering after an off-domain redirect ([#3923](https://github.com/apify/crawlee/issues/3923)) ([9b312ef](https://github.com/apify/crawlee/commit/9b312efd14d607437f6a3865eea8bb67ba4cf05c)), closes [#3921](https://github.com/apify/crawlee/issues/3921)
* **core:** remove `RequestList` persist state listener on `teardown` ([#3846](https://github.com/apify/crawlee/issues/3846)) ([63394ee](https://github.com/apify/crawlee/commit/63394ee4bca950db1ee1678d3ea2b95ce45f95c5))
* **core:** skip malformed cookie fragments in mergeCookies ([#3822](https://github.com/apify/crawlee/issues/3822)) ([b3c3ef2](https://github.com/apify/crawlee/commit/b3c3ef2cab175ec0af4e5777bc6d0979b5e71d40))
* **core:** unblock backpressured sitemap load on persistState ([#3863](https://github.com/apify/crawlee/issues/3863)) ([0b31a8b](https://github.com/apify/crawlee/commit/0b31a8b57cdeee4cead83db6c48febf48b988f6f))
* declare missing dependencies ([#3817](https://github.com/apify/crawlee/issues/3817)) ([5a9a661](https://github.com/apify/crawlee/commit/5a9a661401a3aa895738c79a8310d9cc18e28826))
* filter sitemap-derived URLs by enqueue strategy ([#3797](https://github.com/apify/crawlee/issues/3797)) ([9dc3160](https://github.com/apify/crawlee/commit/9dc3160e6b83c549c29c188b7a88fdc8a612be62))
* **impit-client:** honor redirect handler for session cookie parity ([#3938](https://github.com/apify/crawlee/issues/3938)) ([9371301](https://github.com/apify/crawlee/commit/9371301b21f733a5b5f5c2a092512ec34f511843))
* make crawler terminal status message reliably delivered ([#3733](https://github.com/apify/crawlee/issues/3733)) ([7aed264](https://github.com/apify/crawlee/commit/7aed264f1cd9e993ffab93e314e9c7318b30d43d))
* **memory-storage:** don't increment `handledRequestCount` when updating a handled request ([#3827](https://github.com/apify/crawlee/issues/3827)) ([4739d48](https://github.com/apify/crawlee/commit/4739d4886e3c52f90814b9e5342c4ec2cbaa7587))
* **memory-storage:** prevent storage names from escaping the storage directory ([#3715](https://github.com/apify/crawlee/issues/3715)) ([a04c297](https://github.com/apify/crawlee/commit/a04c29766e3165534a5789d59a3a7bef5a8f5b5a))
* **memory-storage:** use platform basename in key-value store warning ([#3864](https://github.com/apify/crawlee/issues/3864)) ([dcefe01](https://github.com/apify/crawlee/commit/dcefe0196d0ba9bd89770e0a9ff5503295dc274b))
* **playwright,puppeteer:** only intercept requests for non-GET methods or payloads ([#3819](https://github.com/apify/crawlee/issues/3819)) ([1942f1e](https://github.com/apify/crawlee/commit/1942f1ec827cb69953a95abae2086400498b32d8))
* **playwright:** silence transient errors in `enqueueLinksByClickingElements` ([#3732](https://github.com/apify/crawlee/issues/3732)) ([7b98cc3](https://github.com/apify/crawlee/commit/7b98cc350f8ef7bd56bda309b5aff5664dd6fed5)), closes [#3216](https://github.com/apify/crawlee/issues/3216)
* read properties directly from `GotScrapingHttpClient` response ([#3807](https://github.com/apify/crawlee/issues/3807)) ([c673c8d](https://github.com/apify/crawlee/commit/c673c8d7771a43640e7714e929a95df0404ceed2))
* respect falsy transformRequestFunction return in context enqueueLinks ([#3925](https://github.com/apify/crawlee/issues/3925)) ([6cb4fc3](https://github.com/apify/crawlee/commit/6cb4fc392f43e7ad42d56114372eb8ad79aeff74)), closes [#3920](https://github.com/apify/crawlee/issues/3920)
* **sitemap:** recognize sitemap_index as a valid input sitemap url ([#3909](https://github.com/apify/crawlee/issues/3909)) ([eacab5e](https://github.com/apify/crawlee/commit/eacab5e155aeb5dca62cd0c03ef3553be65dd32b))
* **stagehand:** wait for Stagehand to register a page before using it ([#3865](https://github.com/apify/crawlee/issues/3865)) ([238214f](https://github.com/apify/crawlee/commit/238214f9e1586d63837c0b2c7b0a2cfc9a945ae4))
* update handleCloudflareChallenge for new Cloudflare challenge markup ([#3717](https://github.com/apify/crawlee/issues/3717)) ([5baa8bd](https://github.com/apify/crawlee/commit/5baa8bdd7baba927871ed20859405e14312297b6)), closes [#3629](https://github.com/apify/crawlee/issues/3629) [apify/camoufox-js#283](https://github.com/apify/camoufox-js/issues/283) [#3629](https://github.com/apify/crawlee/issues/3629)
* **utils:** advance the CPU tick baseline in systemInfoV2 getCurrentCpuTicks ([#3824](https://github.com/apify/crawlee/issues/3824)) ([bb23800](https://github.com/apify/crawlee/commit/bb238007e34009371807742ba11ecbb9974887b3))
* **utils:** allow apostrophes and asterisks in extracted URLs ([#3790](https://github.com/apify/crawlee/issues/3790)) ([e9baffd](https://github.com/apify/crawlee/commit/e9baffd78ec42079d5c9a8a6e62d43a594ae42c8)), closes [#2755](https://github.com/apify/crawlee/issues/2755)
* **utils:** prevent ReDoS in social email regex ([#3845](https://github.com/apify/crawlee/issues/3845)) ([d6a2b39](https://github.com/apify/crawlee/commit/d6a2b39252c0f00290e75a73def057fd39625525))
* **utils:** prevent sitemap metadata leak across url without loc ([#3795](https://github.com/apify/crawlee/issues/3795)) ([895ed62](https://github.com/apify/crawlee/commit/895ed6264d7eadc5306b92303aeb99bff41cb286))
* **utils:** use article:* selectors for article Open Graph properties ([#3789](https://github.com/apify/crawlee/issues/3789)) ([b660506](https://github.com/apify/crawlee/commit/b6605065fc4d7b692bcdac3ee2004042e8de7457))

### Features

* **core:** infer default-route handler userData from its schema ([#3919](https://github.com/apify/crawlee/issues/3919)) ([37d0ccb](https://github.com/apify/crawlee/commit/37d0ccbb2199bc05620bf4e359b0769bf6f1e05e))
* **impit-client:** add `cacheClients` option to disable `Impit` client reuse ([#3771](https://github.com/apify/crawlee/issues/3771)) ([af9cb8c](https://github.com/apify/crawlee/commit/af9cb8c68c8299e39fd2c40c3ff411e35d8a03f3)), closes [#3769](https://github.com/apify/crawlee/issues/3769) [#3769](https://github.com/apify/crawlee/issues/3769)
* opt-in schema validation of request userData per router label ([#3851](https://github.com/apify/crawlee/issues/3851)) ([c7347f9](https://github.com/apify/crawlee/commit/c7347f94f72980b7c424fb4fc7dc8f990f360d70)), closes [#3747](https://github.com/apify/crawlee/issues/3747) [#3748](https://github.com/apify/crawlee/issues/3748) [#3747](https://github.com/apify/crawlee/issues/3747) [#3082](https://github.com/apify/crawlee/issues/3082)
* support `puppeteer@25` ([#3869](https://github.com/apify/crawlee/issues/3869)) ([1430062](https://github.com/apify/crawlee/commit/1430062c2ba49c32ecb4cd4a14edd2d9a71c7ac3)), closes [#3772](https://github.com/apify/crawlee/issues/3772)
* type-safe router labels via per-label userData map ([#3747](https://github.com/apify/crawlee/issues/3747)) ([1124aca](https://github.com/apify/crawlee/commit/1124aca3c5eb527c097decd7263a6816c411369e)), closes [#3082](https://github.com/apify/crawlee/issues/3082)

### Performance Improvements

* **core:** improve `RequestList` initialization speed on large arrays ([#3754](https://github.com/apify/crawlee/issues/3754)) ([c451c13](https://github.com/apify/crawlee/commit/c451c13a9c9b5c884ce70e0d1257e15597fcb10b)), closes [#3750](https://github.com/apify/crawlee/issues/3750)