py-1.44.0

pola-rs/polarspy-1.44.0Aug 24, 2026by github-actions[bot]

AI Summary

A major feature release introducing a new RemoteEngine architecture, significantly expanding Iceberg support (including schema evolution and V3 deletion vectors), and enhancing SQL capabilities and performance.

Key Highlights

  • Introduction of `RemoteEngine` and a common base class for all engines.
  • Expanded Iceberg support including schema evolution and V3 deletion vectors.
  • Significant SQL improvements including correlated subqueries and conformance.
  • Performance optimizations including HTTP rate limiting and CTE caching.

Breaking Changes

  • Deprecate `rechunk` parameter for all read/scan functions.
  • Deprecate `Expr.rechunk()`.
  • Deprecate `struct.rename_fields()` with an incorrect number of fields.

New Features

  • Support integer fixed-array dot products.
  • SQL correlated in support.
  • Support more join_types in `join_where`.
  • Support Iceberg schema evolution.
  • Support Iceberg table with V3 deletion vectors.
  • Add `struct.drop()` method.
  • Add adaptive HTTP rate-limiter for cloud IO.

Full Release Notes

## ⚠️ Deprecations

- Deprecate `rechunk` parameter for all read/scan functions (#28063)
- Deprecate `Expr.rechunk()` (#28692)
- Deprecate `struct.rename_fields()` with an incorrect number of fields (#28672)

## 🚀 Performance improvements

- Add private env var toggle for HTTP rate limit (#28882)
- Relax default `CloudRetryConfig` for rate limit stability (#28885)
- Elide generic cross join on subquery decorrelation in equality predicates (#28876)
- cache CTE's in SQL layer (#28864)
- Allow CSPE if manual caches are set (#28859)
- Lower unqualified join predicates to inner joins in SQL (#28854)
- Rechunk `mask` for DataFrame `filter` (#28762)
- Improve performance of when/then/otherwise by masking out unevaluated elements (#28498)
- Drop unused projections in filter on streaming engine (#28713)
- Retain Parquet `Partial` metadata on filter (#28737)

## ✨ Enhancements

- Support integer fixed-array dot products (#28829)
- SQL correlated in (#28927)
- Support more join\_types in `join_where` (#28880)
- Support Iceberg schema evolution (#28794)
- SQL conformance (#28494)
- Introduce `RemoteEngine` and a common base class for all engines (#28800)
- Support Iceberg table with V3 deletion vectors in native `scan_iceberg` (#28772)
- Support Iceberg snapshot properties (#28793)
- Expose `sinked_paths_callback` on `sink_parquet` as unstable parameter (#28814)
- Add `infer_schema_files` to CSV inference hint (#28809)
- Add native fixed-array dot (#28504)
- Export query metrics to polars\_cloud (#28757)
- Support Iceberg object storage paths (#28634)
- More CSEE simplification (#28731)
- Improve error in Series constructor with name in context (#28743)
- Add adaptive HTTP rate-limiter for cloud IO (#28591)
- Add `struct.drop()` (#28666)
- Improve error message when CSV name de-duplication fails (#28658)
- Always keep first metadata per source for Parquet (#28661)
- Improve plan-time row estimates for multi-file parquet scans (#28380)
- Expose array and plugin function views in Python visitor (#28635)

## 🐞 Bug fixes

- SQL handle quantified comparisons against a subquery (#28929)
- Fix SQL subquery qualified lowering and cache row-index (#28926)
- Clippy-stable failures (#28919)
- Use uuidv7 for cloud observed queries (#28910)
- Unsoundness in AnyValue::into\_static for structs (#28889)
- Invalidate suborders on projection (#28866)
- Ensure SQL function registry is available in isolated subqueries (#28881)
- Handle null in Expr.is\_not\_nan (#28883)
- Ensure eager `unpivot` column selection aligns with the lazy engine (#28846)
- Fix assertion error / incorrect data load from  parquet written by fastparquet (#28656)
- Fix projection pushdown incorrect `select()` height (#28751)
- Restore `monitoring` as an engine-level parameter (#28821)
- Bincode serialization of IR hints (#28837)
- Retain datetime inference across CSV fallback (#28663)
- Avoid chunk fragmentation on bool agg `min` and `max` (#28789)
- Merge join with coalesce and empty suffix (#28799)
- Wrong function name call in test (#28801)
- Preserve row mapping in nested over expressions (#28767)
- Restore CI compatibility with NumPy 2.5.2 (#28782)
- Do not deduplicate inherently nondeterministic plan nodes (#28768)
- Fix OOC spilling of categoricals (#28761)
- Correct partitionable filter handling for pivot (#28749)
- Small typo in docstring (#28763)
- Avoid chunk fragmentation on bool agg `any` and `all` (#28754)
- Ignore nans in (min|max)\_horizontal (#28710)
- Correct function display for str.replace\_all (#28742)
- Exclude debug tests from nodebug streaming run (#28747)
- Convert predicate series dtype during prefiltered parquet reading (#28711)
- Do not remove nested caches when performing predicate pushdown past a cache node (#28740)
- Correct handling of nulls at the start of business day functions (#28703)
- Do not panic when exceeding Parquet metadata scan cache (#28721)
- Remove cryptographic hashing from CSPE (#28716)
- Unsoundness in rayon block\_on (#28709)
- Check for (shallow) IR node equality on top of hashing in CSPE (#28506)
- Don't hand IO plugins a predicate they cannot evaluate (#28643)
- Raise on overflow in Decimal sum instead of silently wrapping (#28688)
- Preserve nulls when importing Arrow maps (#28680)
- Propagate null `by` column values in `rolling_*_by` (#27367)
- Fix self-referencing `field` in `struct.with_fields` with `over` (#28678)
- Fix Arrow buffer offset for `Utf8` and `Binary` (#28662)
- Clamp group-by slice offset (#28579)
- Release the GIL while resolving the schema in `__arrow_c_stream__` (#28642)
- Fix data corruption on Series from nested Arrow `LargeList` (#28632)

## 📖 Documentation

- Interlink join docstrings (#28907)
- Migration guide for 2.0 release (#28788)
- Clarify `read_database_uri` requirements for SQLAlchemy (#28366)
- Update engine tags (#28852)
- Clarify "that that side" comment in asof\_join (#28847)
- Update mage.ai documentation link (#28813)
- Add engine tags (#28708)
- Document that arg\_extremum functions return any index on ties (#28691)
- Fix Polars Cloud API reference link (#28693)

## 🛠️ Other improvements

- Bump runtime toolchains (#28924)
- Add min-publish-age for cargo (#28906)
- Tighten join invariants DSL (#28913)
- Bump Rust toolchain (#28900)
- Ensure that predicates are pushed down to the scans (#28857)
- Clarify "that that side" comment in asof\_join (#28847)
- Restore `monitoring` as an engine-level parameter (#28821)
- Remove workaround for inconsistent CSE (#28834)
- Introduce `RemoteEngine` and a common base class for all engines (#28800)
- Add row count and file byte size to sinked paths callback (#28522)
- Use canonical expression IDs in CSEE (#28786)
- Enforce 100% type coverage in CI with Pyrefly (#28795)
- Attribute annotations for `CredentialProviderAWS` and `CredentialProviderAzure` (#28790)
- Attribute annotations for `ScanCastOptions` (#27906)
- Disable credentials in upload chunk size tests (#28745)
- Improve backtrace path formatting (#28732)
- Exclude debug tests from nodebug streaming run (#28747)
- Mark `test_grouped_agg_parametric` as slow (and add param ids) (#28715)
- Fix first `expect` in `test_group_by_arg_max_boolean_26978` (#28690)
- Fix the name of `struct.drop()` in `rename_fields()` deprecation message (#28687)
- Add `dist=loadgroup` to the pytest config (#28676)
- Enable a lot of `may_fail_auto_streaming` tests (#28675)
- Re-enable `test_extension()` for streaming engine (#28611)
- Update `test_hive_join_rewrite_semi_join` test to work with streaming engine (#28610)

Thank you to all our contributors for making this release possible!
@0guban0v, @JakubValtar, @Joosboy, @Kevin-Patyk, @MatteoPossamai, @NicoOhR, @TNieuwdorp, @VedantMadane, @aarushkandukoori, @alexander-beedie, @azimafroozeh, @borchero, @carnarez, @dancsi, @dependabot[bot], @dsprenkels, @gautamvarmadatla, @jonasdedden, @jorenham, @kdn36, @lzcmian, @malhotrashivam, @matthewbayer, @mikhail5555, @nameexhaustion, @orlp, @ritchie46, @subotac and [dependabot[bot]](https://github.com/apps/dependabot)