py-1.44.0
pola-rs/polarspy-1.44.0Aug 24, 2026by github-actions[bot]
AI Summary
A major feature release introducing a new RemoteEngine architecture, significantly expanding Iceberg support (including schema evolution and V3 deletion vectors), and enhancing SQL capabilities and performance.
Key Highlights
- Introduction of `RemoteEngine` and a common base class for all engines.
- Expanded Iceberg support including schema evolution and V3 deletion vectors.
- Significant SQL improvements including correlated subqueries and conformance.
- Performance optimizations including HTTP rate limiting and CTE caching.
Breaking Changes
- Deprecate `rechunk` parameter for all read/scan functions.
- Deprecate `Expr.rechunk()`.
- Deprecate `struct.rename_fields()` with an incorrect number of fields.
New Features
- Support integer fixed-array dot products.
- SQL correlated in support.
- Support more join_types in `join_where`.
- Support Iceberg schema evolution.
- Support Iceberg table with V3 deletion vectors.
- Add `struct.drop()` method.
- Add adaptive HTTP rate-limiter for cloud IO.
Full Release Notes
## ⚠️ Deprecations - Deprecate `rechunk` parameter for all read/scan functions (#28063) - Deprecate `Expr.rechunk()` (#28692) - Deprecate `struct.rename_fields()` with an incorrect number of fields (#28672) ## 🚀 Performance improvements - Add private env var toggle for HTTP rate limit (#28882) - Relax default `CloudRetryConfig` for rate limit stability (#28885) - Elide generic cross join on subquery decorrelation in equality predicates (#28876) - cache CTE's in SQL layer (#28864) - Allow CSPE if manual caches are set (#28859) - Lower unqualified join predicates to inner joins in SQL (#28854) - Rechunk `mask` for DataFrame `filter` (#28762) - Improve performance of when/then/otherwise by masking out unevaluated elements (#28498) - Drop unused projections in filter on streaming engine (#28713) - Retain Parquet `Partial` metadata on filter (#28737) ## ✨ Enhancements - Support integer fixed-array dot products (#28829) - SQL correlated in (#28927) - Support more join\_types in `join_where` (#28880) - Support Iceberg schema evolution (#28794) - SQL conformance (#28494) - Introduce `RemoteEngine` and a common base class for all engines (#28800) - Support Iceberg table with V3 deletion vectors in native `scan_iceberg` (#28772) - Support Iceberg snapshot properties (#28793) - Expose `sinked_paths_callback` on `sink_parquet` as unstable parameter (#28814) - Add `infer_schema_files` to CSV inference hint (#28809) - Add native fixed-array dot (#28504) - Export query metrics to polars\_cloud (#28757) - Support Iceberg object storage paths (#28634) - More CSEE simplification (#28731) - Improve error in Series constructor with name in context (#28743) - Add adaptive HTTP rate-limiter for cloud IO (#28591) - Add `struct.drop()` (#28666) - Improve error message when CSV name de-duplication fails (#28658) - Always keep first metadata per source for Parquet (#28661) - Improve plan-time row estimates for multi-file parquet scans (#28380) - Expose array and plugin function views in Python visitor (#28635) ## 🐞 Bug fixes - SQL handle quantified comparisons against a subquery (#28929) - Fix SQL subquery qualified lowering and cache row-index (#28926) - Clippy-stable failures (#28919) - Use uuidv7 for cloud observed queries (#28910) - Unsoundness in AnyValue::into\_static for structs (#28889) - Invalidate suborders on projection (#28866) - Ensure SQL function registry is available in isolated subqueries (#28881) - Handle null in Expr.is\_not\_nan (#28883) - Ensure eager `unpivot` column selection aligns with the lazy engine (#28846) - Fix assertion error / incorrect data load from parquet written by fastparquet (#28656) - Fix projection pushdown incorrect `select()` height (#28751) - Restore `monitoring` as an engine-level parameter (#28821) - Bincode serialization of IR hints (#28837) - Retain datetime inference across CSV fallback (#28663) - Avoid chunk fragmentation on bool agg `min` and `max` (#28789) - Merge join with coalesce and empty suffix (#28799) - Wrong function name call in test (#28801) - Preserve row mapping in nested over expressions (#28767) - Restore CI compatibility with NumPy 2.5.2 (#28782) - Do not deduplicate inherently nondeterministic plan nodes (#28768) - Fix OOC spilling of categoricals (#28761) - Correct partitionable filter handling for pivot (#28749) - Small typo in docstring (#28763) - Avoid chunk fragmentation on bool agg `any` and `all` (#28754) - Ignore nans in (min|max)\_horizontal (#28710) - Correct function display for str.replace\_all (#28742) - Exclude debug tests from nodebug streaming run (#28747) - Convert predicate series dtype during prefiltered parquet reading (#28711) - Do not remove nested caches when performing predicate pushdown past a cache node (#28740) - Correct handling of nulls at the start of business day functions (#28703) - Do not panic when exceeding Parquet metadata scan cache (#28721) - Remove cryptographic hashing from CSPE (#28716) - Unsoundness in rayon block\_on (#28709) - Check for (shallow) IR node equality on top of hashing in CSPE (#28506) - Don't hand IO plugins a predicate they cannot evaluate (#28643) - Raise on overflow in Decimal sum instead of silently wrapping (#28688) - Preserve nulls when importing Arrow maps (#28680) - Propagate null `by` column values in `rolling_*_by` (#27367) - Fix self-referencing `field` in `struct.with_fields` with `over` (#28678) - Fix Arrow buffer offset for `Utf8` and `Binary` (#28662) - Clamp group-by slice offset (#28579) - Release the GIL while resolving the schema in `__arrow_c_stream__` (#28642) - Fix data corruption on Series from nested Arrow `LargeList` (#28632) ## 📖 Documentation - Interlink join docstrings (#28907) - Migration guide for 2.0 release (#28788) - Clarify `read_database_uri` requirements for SQLAlchemy (#28366) - Update engine tags (#28852) - Clarify "that that side" comment in asof\_join (#28847) - Update mage.ai documentation link (#28813) - Add engine tags (#28708) - Document that arg\_extremum functions return any index on ties (#28691) - Fix Polars Cloud API reference link (#28693) ## 🛠️ Other improvements - Bump runtime toolchains (#28924) - Add min-publish-age for cargo (#28906) - Tighten join invariants DSL (#28913) - Bump Rust toolchain (#28900) - Ensure that predicates are pushed down to the scans (#28857) - Clarify "that that side" comment in asof\_join (#28847) - Restore `monitoring` as an engine-level parameter (#28821) - Remove workaround for inconsistent CSE (#28834) - Introduce `RemoteEngine` and a common base class for all engines (#28800) - Add row count and file byte size to sinked paths callback (#28522) - Use canonical expression IDs in CSEE (#28786) - Enforce 100% type coverage in CI with Pyrefly (#28795) - Attribute annotations for `CredentialProviderAWS` and `CredentialProviderAzure` (#28790) - Attribute annotations for `ScanCastOptions` (#27906) - Disable credentials in upload chunk size tests (#28745) - Improve backtrace path formatting (#28732) - Exclude debug tests from nodebug streaming run (#28747) - Mark `test_grouped_agg_parametric` as slow (and add param ids) (#28715) - Fix first `expect` in `test_group_by_arg_max_boolean_26978` (#28690) - Fix the name of `struct.drop()` in `rename_fields()` deprecation message (#28687) - Add `dist=loadgroup` to the pytest config (#28676) - Enable a lot of `may_fail_auto_streaming` tests (#28675) - Re-enable `test_extension()` for streaming engine (#28611) - Update `test_hive_join_rewrite_semi_join` test to work with streaming engine (#28610) Thank you to all our contributors for making this release possible! @0guban0v, @JakubValtar, @Joosboy, @Kevin-Patyk, @MatteoPossamai, @NicoOhR, @TNieuwdorp, @VedantMadane, @aarushkandukoori, @alexander-beedie, @azimafroozeh, @borchero, @carnarez, @dancsi, @dependabot[bot], @dsprenkels, @gautamvarmadatla, @jonasdedden, @jorenham, @kdn36, @lzcmian, @malhotrashivam, @matthewbayer, @mikhail5555, @nameexhaustion, @orlp, @ritchie46, @subotac and [dependabot[bot]](https://github.com/apps/dependabot)