py-1.28.0

pola-rs/polarspy-1.28.0Apr 26, 2025by github-actions[bot]

AI Summary

Significant performance improvements for the streaming engine and new features including GPU support for sinks and enhanced rolling statistics like kurtosis.

Key Highlights

  • Add GPU support to sink APIs
  • Add `rolling_kurtosis` and `rolling_skew` kernels
  • Make streaming dispatch public
  • Performance improvements on new streaming group_by

New Features

  • Add `backward_fill` / `forward_fill` to Series
  • Add `rolling_kurtosis`
  • Support Cast in IO plugin predicates
  • Add `.sort(nulls_last=True)` to booleans, categoricals, enums
  • Add rolling min/max for temporals
  • Support literal:list agg

Full Release Notes

## 🚀 Performance improvements

- Lower Expr.(n\_)unique to group\_by on streaming engine (#22420)
- Chunk huge munmap calls (#22414)
- Add single-key variants of streaming group\_by (#22409)
- Improve accumulate\_dataframes\_vertical performance (#22399)
- Use optimize rolling\_quantile with varying window sizes (#22353)
- Dedicated `rolling_skew` kernel (#22333)
- Call large munmap's in background thread (#22329)
- New streaming group\_by implementation (#22285)
- Patch jemalloc to not purge huge allocs eagerly if we have background threads (#22318)
- Turn on `parallel=prefiltered` by default for new streaming (#22190)

## ✨ Enhancements

- When reporting unexpected types in errors, module-qualify the typename (#22390)
- Add Series `backward_fill` / `forward_fill` (#22360)
- Add GPU support to sink\_\* APIs (#20940)
- Changed mapping type from `dict` to `Mapping` (#19400) (#19436)
- Make streaming dispatch public (#22347)
- Add `rolling_kurtosis` (#22335)
- Support Cast in IO plugin predicates (#22317)
- Add `.sort(nulls_last=True)` to booleans, categoricals and enums (#22300)
- Add rolling min/max for temporals (#22271)
- Support literal:list agg (#22249)
- Support running Polars SQL queries against any objects implementing the PyCapsule interface (#22235)
- Support `implode + agg` (#22230)
- Dispatch scans to new-streaming by default (#22153)

## 🐞 Bug fixes

- Ensure `write_excel` correctly preserves null values in nested dtype data on export (#22379)
- Panic when visualizing streaming physical plan with joins (#22404)
- Fix incorrect filter after `LazyFrame.rename().select()` (#22380)
- Fix `select(len())` performance regression (#22363)
- Handle pytz named timezone in `lit` (#21785)
- Don't leak state during prefill CSE cache (#22341)
- Maintain float32 type in partitioned group-by (#22340)
- Resolve streaming panic on multiple `merge_sorted` (#22205)
- Fix ndjson nested types (#22325)
- Fix nested datetypes in ndjson (#22321)
- Check matching lengths for `pl.corr` (#22305)
- Move type coercion for `pl.duration` to planner (#22304)
- Check dtype to avoid panic with mixed types in min/max\_horizontal (#21857)
- Coalesce correct column for new streaming full join (#22301)
- Don't collect `NaN` from Parquet Statistics (#22294)
- Set revmap for empty `AnyValue` to `Series` (#22293)
- Add an `__all__` entry to internal type definition module (#22254)
- Datetime parser was incorrectly parsing 8-digit fractional seconds when format specified to expect 9 (#22180)
- More robust `str → date` conversion when reading from spreadsheet (#22276)
- Deprecate using `is_in` with 2 equal types and mark as elementwise (#22178)
- Duplicate key column name in streaming group\_by due to CSE (#22280)
- Raise `ColumnNotFoundError` for missing columns in `join_where` (#22268)
- Parquet filters for logical types and operations (#22253)
- Ensure floating-point accuracy in `hist` (#22245)
- Check matching key datatypes for new streaming joins (#22247)
- Incorrect length BinaryArray/ListBuilder (#22227)

## 📖 Documentation

- Update docs for schema arg in scan\_csv to match read\_csv (#22357)
- Update `pl.when` documentation (#22345)
- Add missing `is_business_day` to documentation reference (#22338)
- Improve interpolation documentation to clarify behavior of null values (#22274)

## 🛠️ Other improvements

- Install pytorch for 3.13 on Windows (#22356)
- Make interpolate fix more robust (#22421)
- Fix interpolate test (#22417)
- Reduce hot table size in debug mode (#22400)
- Replace intrinsic with non-intrinsic (#22401)
- Make streaming dispatch public (#22347)
- Update rustc to 'nightly-2025-04-19' (#22342)
- Update mozilla-actions/sccache-action (#22319)
- Purge old parquet and scan code (#22226)
- Add an `__all__` entry to internal type definition module (#22254)
- Add online skew/kurtosis algorithm for future use in rolling kernels (#22261)
- Add Polars Cloud 0.0.7 release notes (#22223)
- Change format name from list to implode (#22240)
- Make other parallel parquet modes filter afterwards (#22228)
- Close async reader issues (#22224)
- Add BinaryArrayBuilder (#22225)

Thank you to all our contributors for making this release possible!
@DavideCanton, @JakubValtar, @Jesse-Bakker, @MarcoGorelli, @NeejWeej, @Shoeboxam, @adamreeve, @alexander-beedie, @axellpadilla, @cmdlineluser, @coastalwhite, @d-reynol, @dongchao-1, @florian-klein, @kdn36, @math-hiyoko, @mcrumiller, @mroeschke, @nameexhaustion, @orlp, @ritchie46, @stijnherfst and @yiteng-guo