py-1.28.0
pola-rs/polarspy-1.28.0Apr 26, 2025by github-actions[bot]
AI Summary
Significant performance improvements for the streaming engine and new features including GPU support for sinks and enhanced rolling statistics like kurtosis.
Key Highlights
- Add GPU support to sink APIs
- Add `rolling_kurtosis` and `rolling_skew` kernels
- Make streaming dispatch public
- Performance improvements on new streaming group_by
New Features
- Add `backward_fill` / `forward_fill` to Series
- Add `rolling_kurtosis`
- Support Cast in IO plugin predicates
- Add `.sort(nulls_last=True)` to booleans, categoricals, enums
- Add rolling min/max for temporals
- Support literal:list agg
Full Release Notes
## 🚀 Performance improvements - Lower Expr.(n\_)unique to group\_by on streaming engine (#22420) - Chunk huge munmap calls (#22414) - Add single-key variants of streaming group\_by (#22409) - Improve accumulate\_dataframes\_vertical performance (#22399) - Use optimize rolling\_quantile with varying window sizes (#22353) - Dedicated `rolling_skew` kernel (#22333) - Call large munmap's in background thread (#22329) - New streaming group\_by implementation (#22285) - Patch jemalloc to not purge huge allocs eagerly if we have background threads (#22318) - Turn on `parallel=prefiltered` by default for new streaming (#22190) ## ✨ Enhancements - When reporting unexpected types in errors, module-qualify the typename (#22390) - Add Series `backward_fill` / `forward_fill` (#22360) - Add GPU support to sink\_\* APIs (#20940) - Changed mapping type from `dict` to `Mapping` (#19400) (#19436) - Make streaming dispatch public (#22347) - Add `rolling_kurtosis` (#22335) - Support Cast in IO plugin predicates (#22317) - Add `.sort(nulls_last=True)` to booleans, categoricals and enums (#22300) - Add rolling min/max for temporals (#22271) - Support literal:list agg (#22249) - Support running Polars SQL queries against any objects implementing the PyCapsule interface (#22235) - Support `implode + agg` (#22230) - Dispatch scans to new-streaming by default (#22153) ## 🐞 Bug fixes - Ensure `write_excel` correctly preserves null values in nested dtype data on export (#22379) - Panic when visualizing streaming physical plan with joins (#22404) - Fix incorrect filter after `LazyFrame.rename().select()` (#22380) - Fix `select(len())` performance regression (#22363) - Handle pytz named timezone in `lit` (#21785) - Don't leak state during prefill CSE cache (#22341) - Maintain float32 type in partitioned group-by (#22340) - Resolve streaming panic on multiple `merge_sorted` (#22205) - Fix ndjson nested types (#22325) - Fix nested datetypes in ndjson (#22321) - Check matching lengths for `pl.corr` (#22305) - Move type coercion for `pl.duration` to planner (#22304) - Check dtype to avoid panic with mixed types in min/max\_horizontal (#21857) - Coalesce correct column for new streaming full join (#22301) - Don't collect `NaN` from Parquet Statistics (#22294) - Set revmap for empty `AnyValue` to `Series` (#22293) - Add an `__all__` entry to internal type definition module (#22254) - Datetime parser was incorrectly parsing 8-digit fractional seconds when format specified to expect 9 (#22180) - More robust `str → date` conversion when reading from spreadsheet (#22276) - Deprecate using `is_in` with 2 equal types and mark as elementwise (#22178) - Duplicate key column name in streaming group\_by due to CSE (#22280) - Raise `ColumnNotFoundError` for missing columns in `join_where` (#22268) - Parquet filters for logical types and operations (#22253) - Ensure floating-point accuracy in `hist` (#22245) - Check matching key datatypes for new streaming joins (#22247) - Incorrect length BinaryArray/ListBuilder (#22227) ## 📖 Documentation - Update docs for schema arg in scan\_csv to match read\_csv (#22357) - Update `pl.when` documentation (#22345) - Add missing `is_business_day` to documentation reference (#22338) - Improve interpolation documentation to clarify behavior of null values (#22274) ## 🛠️ Other improvements - Install pytorch for 3.13 on Windows (#22356) - Make interpolate fix more robust (#22421) - Fix interpolate test (#22417) - Reduce hot table size in debug mode (#22400) - Replace intrinsic with non-intrinsic (#22401) - Make streaming dispatch public (#22347) - Update rustc to 'nightly-2025-04-19' (#22342) - Update mozilla-actions/sccache-action (#22319) - Purge old parquet and scan code (#22226) - Add an `__all__` entry to internal type definition module (#22254) - Add online skew/kurtosis algorithm for future use in rolling kernels (#22261) - Add Polars Cloud 0.0.7 release notes (#22223) - Change format name from list to implode (#22240) - Make other parallel parquet modes filter afterwards (#22228) - Close async reader issues (#22224) - Add BinaryArrayBuilder (#22225) Thank you to all our contributors for making this release possible! @DavideCanton, @JakubValtar, @Jesse-Bakker, @MarcoGorelli, @NeejWeej, @Shoeboxam, @adamreeve, @alexander-beedie, @axellpadilla, @cmdlineluser, @coastalwhite, @d-reynol, @dongchao-1, @florian-klein, @kdn36, @math-hiyoko, @mcrumiller, @mroeschke, @nameexhaustion, @orlp, @ritchie46, @stijnherfst and @yiteng-guo