rs-0.48.0
pola-rs/polarsrs-0.48.0May 20, 2025by github-actions[bot]
AI Summary
Rust release synchronized with Python 1.30.0. Focuses on the new streaming engine (cross-join, multiscan) and various bug fixes.
Key Highlights
- Add streaming cross-join node
- Implemented `list.filter`
- Improved time-string parsing and inference
- Implement `LazyFrame.match_to_schema`
Breaking Changes
- Use a wrapper struct to store time zone (#22523)
New Features
- Implemented `list.filter`
- Add `nulls_equal` flag to `list/arr.contains`
- Implement `LazyFrame.match_to_schema`
- Support optimised init from non-dict Mapping objects
- Add options to write Parquet field metadata
Full Release Notes
## 💥 Breaking changes - Use a wrapper struct to store time zone (#22523) ## 🚀 Performance improvements - Allow predicate passing set\_sorted (#22797) - Increase default cross-file parallelism limit for new-streaming multiscan (#22700) - Add elementwise execution mode for `list.eval` (#22715) - Support optimised init from non-dict `Mapping` objects in `from_records` and frame/series constructors (#22638) - Add streaming cross-join node (#22581) - Switch off `maintain_order` in group-by followed by sort (#22492) ## ✨ Enhancements - Format named functions (#22831) - Implemented `list.filter` (#22749) - Support binaryoffset in search sorted (#22786) - Add `nulls_equal` flag to `list/arr.contains` (#22773) - Allow named opaque functions for serde (#22734) - Implement `LazyFrame.match_to_schema` (#22726) - Improved time-string parsing and inference (generally, and via the SQL interface) (#22606) - Allow for `.over` to be called without `partition_by` (#22712) - Support `AnyValue` translation from `PyMapping` values (#22722) - Support optimised init from non-dict `Mapping` objects in `from_records` and frame/series constructors (#22638) - Add options to write Parquet field metadata (#22652) - Allow casting `List<UInt8>` to `Binary` (#22611) - Allow setting of regex size limit using `POLARS_REGEX_SIZE_LIMIT` (#22651) ## 🐞 Bug fixes - Fix reverse list type (#22832) - Add type equality checking for relevant methods (#22802) - Invalid output for `fill_null` after `when.then` on structs (#22798) - Don't panic for cross join with misaligned chunking (#22799) - Panic on quantile over nulls in rolling window (#22792) - Respect BinaryOffset metadata (#22785) - Correct the output order of `PartitionByKey` and `PartitionParted` (#22778) - Fallback to non-strict casting for deprecated casts (#22760) - Clippy on new stable version (#22771) - Handle sliced out remainder for bitmaps (#22759) - Don't merge `Enum` categories on append (#22765) - Fix unnest() not working on empty struct columns (#22391) - Correct name in `unnest` error message (#22740) - Properly account for nulls in the `is_not_nan` check made in `drop_nans` (#22707) - Incorrect result from SQL `count(*)` with `partition by` (#22728) - Fix deadlock joining scanned tables with low thread count (#22672) - Don't allow deserializing incompatible DSL (#22644) - Incorrect null dtype from binary ops in empty group\_by (#22721) - Don't mark `str.replace_many` with Mapping as deprecated (#22697) - Gzip has maximum compression of 9, not 10 (#22685) - Fix predicate pushdown of fallible expressions (#22669) - Fix `index out of bounds` panic when scanning hugging face (#22661) - Fix polars crate not compiling when lazy feature enabled (#22655) - Panic on `group_by` with literal and empty rows (#22621) - Return input instead of panicking if empty subset in `drop_nulls()` and `drop_nans()` (#22469) - Bump argminmax to 0.6.3 (#22649) - DSL version deserialization endianness (#22642) - Fix nested dtype row encoding (#22557) - Allow Expr.round() to be called on integer dtypes (#22622) - Fix panic when filtering based on row index column in parquet (#22616) - WASM and PyOdide compile (#22613) - Resolve `get()` SchemaMismatch panic (#22350) ## 📖 Documentation - Add pre-release policy (#22808) - Fix broken link to service account page in Polars Cloud docs (#22762) - Rework documentation for `drop`/`fill` for nulls/nans (#22657) ## 📦 Build system - Patch pyo3 to disable recompilation (#22796) ## 🛠️ Other improvements - Update Polars Rust versions (#22834) - Cleanup `polars-python` lifetimes (#22548) - Fix `nix build` (#22809) - Fix flake.nix to work on macos (#22803) - Remove unused dependencies in `polars-arrow` (#22806) - Unused variables on release build (#22800) - Update cloud docs (#22624) - Add proptest implementations for all Array types (#22711) - Dispatch `.write_*` to `.lazy().sink_*(engine='in-memory')` (#22582) - Move to all optimization flags to `QueryOptFlags` (#22680) - Add test for `str.replace_many` (#22615) - Stabilize `sink_*` (#22643) - Add proptest for row-encode (#22626) - Emphasize PolarsDataType::get\_dtype is static-only (#22648) - Use named fields for Logical (#22647) - Update rust version in nix flake (#22627) - Add a nix flake with a devShell and package (#22246) - Use a wrapper struct to store time zone (#22523) - Add `proptest` testing for for parquet decoding kernels (#22608) Thank you to all our contributors for making this release possible! @IvanIsCoding, @JakubValtar, @Julian-J-S, @LucioFranco, @MarcoGorelli, @WH-2099, @alexander-beedie, @borchero, @bschoenmaeckers, @cmdlineluser, @coastalwhite, @etiennebacher, @florian-klein, @itamarst, @kdn36, @nameexhaustion, @nikaltipar, @orlp, @pavelzw, @r-brink, @ritchie46, @stijnherfst, @teotwaki, @timkpaine and @wence-