py-1.26.0

pola-rs/polarspy-1.26.0Mar 23, 2025by github-actions[bot]

AI Summary

Performance optimizations for joins and gather, enabled Transparent Huge Pages (THP) by default, and added support for IO plugins in the new streaming engine.

Key Highlights

  • Enable Transparent Huge Pages (THP) by default
  • Use views for binary hash tables
  • Add support for io-plugins in new streaming

New Features

  • Add support for io-plugins in new-streaming
  • Add `PartitionParted`
  • Add `explain_all`
  • Add `PartitionByKey` for new streaming sinks
  • Allow to use `repeat_by` with (nested) lists and structs

Full Release Notes

## 🚀 Performance improvements

- Use views for binary hash tables and add single-key binary variant (#21872)
- Avoid rechunking in gather (#21876)
- Switch ahash for foldhash (#21852)
- Put THP behind feature flag (#21853)
- Enable THP by default (#21829)
- Improve join performance for expanding joins (#21821)
- Use binary\_search instead of contains in business-day functions (#21775)

## ✨ Enhancements

- Add support for io-plugins in new-streaming (#21870)
- Add `PartitionParted` (#21788)
- Minor improvements to EXPLAIN plan output (#21822)
- Add `explain_all` (#21797)
- Allow to use `repeat_by` with (nested) lists and structs (#21206)

## 🐞 Bug fixes

- Fix `DataFrame.nan_to_null` work for tuple (#21861)
- Allow `pivot` on empty frame for all integer index dtypes (#21890)
- Null panic on decimal aggregate (#21873)
- Join with categoricals on new-streaming engine (#21825)
- Fix div 0 partitioned group-by (#21842)
- Incorrect quote check in CSV parser (#21826)
- Add option to use relative paths for plugin libraries (#21675)
- Respect header separator in `sink_csv` (#21814)
- Deprecation of `streaming=False` (#21813)
- Fix collect\_all type-coercion (#21810)
- Memory leaks in SharedStorage (#21798)
- Make `None` refer to `uncompressed` in `sink_ipc` (#21786)

## 📖 Documentation

- Add sources and sinks to user-guide (#21780)

## 🛠️ Other improvements

- Change dynamic literals to be separate category (#21849)
- Add POLARS\_TIMEOUT\_MS for timing out slow Polars tests (#21887)
- Disable --dist loadgroup in pytest (#21885)
- Fix refcount assert being messed up by pytest assertion magic (#21884)
- Add env vars to configure new-streaming buffer sizes (#21818)

Thank you to all our contributors for making this release possible!
@Kevin-Patyk, @MarcoGorelli, @alexander-beedie, @anath2, @borchero, @coastalwhite, @dongchao-cn, @kgv, @mcrumiller, @nameexhaustion, @orlp and @ritchie46