py-1.32.1
pola-rs/polarspy-1.32.1Aug 6, 2025by github-actions[bot]
AI Summary
Python Polars 1.32.1 focuses on streaming engine lowering of operations and Iceberg/Delta Lake pre-filtering optimizations.
Key Highlights
- Streaming engine lowering for extend_constant and repeat
- rle_id to native streaming node
- Iceberg/Delta Lake pre-filtering decode
- PyCapsule schema support
New Features
- Lower rle_id to native streaming node
- dt.days_in_month function
- PyCapsule schema support
- repeat_by for Array and Null
Full Release Notes
## 🚀 Performance improvements - Optimise `BytecodeParser` usage from `warn_on_inefficient_map` (#23809) - Lower extend\_constant to the streaming engine (#23824) - Lower pl.repeat to streaming engine (#23804) - Remove redundant clone (#23771) ## ✨ Enhancements - Lower `rle_id` to a native streaming node (#23894) - Pass `endpoint_url` loaded from `CredentialProviderAWS` to `scan/write_delta` (#23812) - Dispatch `scan_iceberg` to native by default (#23912) - Lower `unique_counts` and `value_counts` to streaming engine (#23890) - Support initializing from `__arrow_c_schema__` protocol in `pl.Schema` (#23879) - Better handle broken local package environment in `show_versions` (#23885) - Implement `dt.days_in_month` function (#23119) - Making `Expr.rolling_*_by` methods available to `pl.Series` (#23742) - Fix errors on native `scan_iceberg` (#23811) - Reinterpret binary data to fixed size numerical array (#22840) - Make `rolling_map` serializable (#23848) - Ensure `CachingCredentialProvider` returns copied credentials dict (#23817) - Change typing for `.remote()` from `LazyFrameExt` to `LazyFrameRemote` (#23825) - Implement `repeat_by` for `Array` and `Null` (#23794) - Add DeprecationWarning on passing physical ordering to Categorical (#23779) - Pre-filtered decode and row group skipping with Iceberg / Delta / scans with cast options (#23792) - Update `BytecodeParser` opcode awareness for upcoming Python 3.14 (#23782) ## 🐞 Bug fixes - Categorical namespace functions fail on `Enum` columns (#23925) - Properly set sumwise complete on filter for missing columns (#23877) - Restore Arrow-FFI-based Python\<->Rust conversion in pyo3-polars (#23881) - Group By with filters (#23917) - Fix `read_csv` ignoring Decimal schema for header-only data (#23886) - Ensure `collect()` native Iceberg always scans latest when no `snapshot_id` is given (#23907) - Writing List(Array) columns to JSON without panic (#23875) - Fill Iceberg missing fields with partition values if present in metadata (#23900) - Create file for streaming sink even if unspawned (#23672) - Update cloud testing environment (#23908) - Parquet filtering on multiple RGs with literal predicate (#23903) - Incorrect datatype passed to libc::write (#23904) - Properly feature gate TZ\_AWARE\_RE usage (#23888) - Improve identification of "non group-key" aggregates in SQL `GROUP BY` queries (#23191) - Spawning tokio task outside reactor (#23884) - Correctly raise DuplicateError on asof\_join with suffix="" (#23864) - Fix errors on native `scan_iceberg` (#23811) - Fix index out of bounds panic filtering parquet (#23850) - Fix error on empty range requests (#23844) - Fix handling of hive partitioning `hive_start_idx` parameter (#23843) - Allow encoding of `pl.Enum` with smaller physicals (#23829) - Filter sorted flag from physical in CategoricalChunked (#23827) - Remove accidental todo! in repeat node (#23822) - Make `meta.pop` operate on `Expr` only (#23808) - Stack overflow in `DslPlan` serde (#23801) - Clear credentials cached in Python when rebuilding object store (#23756) - Datetime selectors with mixed timezone info (#23774) - Support i128 in asof join (#23770) - Remove sleep for credential refresh (#23768) ## 📖 Documentation - Improve StackOverflow links in contributing guide (#23895) - Fix `pyo3` documentation page link (#23839) - Document the pureness requirements of udfs (#23787) - Correct the `name.*` methods on their removal of aliases (#23773) ## 📦 Build system - Workaround for pyiceberg `make requirements` on Python 3.13 (#23810) - Add pyiceberg to dev dependencies (#23791) ## 🛠️ Other improvements - Ensure `clippy` and `rustfmt` run in CI when changing `pyo3-polars` (#23930) - Fix pyo3-polars proc-macro re-exports (#23918) - Rewrite `evaluate_on_groups` for `.gather` / `.get` (#23700) - Move Python C API to `python-polars` (#23876) - Improve/fix internal `LRUCache` implementation and move into "\_utils" module (#23813) - Relax constraint on maximum Python version for `numba` (#23838) - Automatically tag PRs mentioning "SQL" with the appropriate label (#23816) - Update `typos` package (#23818) - Fix typos path (#23803) - Remove `deserialize_with_unknown_fields` (#23802) - Add pyiceberg to dev dependencies (#23791) - Remove old schema file (#23798) - Mark more tests as ready for cloud (#23743) - Reduce required deps for pyo3-polars (#23761) Thank you to all our contributors for making this release possible! @JakubValtar, @Kevin-Patyk, @Liyixin95, @alexander-beedie, @cgevans, @cmdlineluser, @coastalwhite, @eitsupi, @gfvioli, @itamarst, @jimmmmmmmmmmmy, @kdn36, @math-hiyoko, @mcrumiller, @mpasa, @mrkn, @nameexhaustion, @orlp, @pka, @pomo-mondreganto, @ritchie46 and @stijnherfst