py-1.9.0

pola-rs/polarspy-1.9.0Oct 1, 2024by github-actions[bot]

AI Summary

Python release focusing on performance improvements, bitwise operations, and new Parquet options.

Key Highlights

  • Use List's TotalEqKernel
  • Bitwise operations / aggregations
  • Support `schema` arg in `read/scan_parquet()`

New Features

  • Bitwise operations / aggregations
  • Allow `insert_column` to take expressions
  • Improved error message DSL -> IR resolving
  • Add `strict` param to eager/lazy frame "rename"
  • Support `schema` arg in `read/scan_parquet()`
  • Add `include_file_paths` parameter to `read_parquet`
  • Add `allow_missing_columns` option to `read/scan_parquet`
  • Drop python 3.8 support
  • Use FFI to extract Series from different Polars binaries
  • Allow for zero-width fixed size lists

Full Release Notes

## 🚀 Performance improvements

- Use List's TotalEqKernel (#18984)

## ✨ Enhancements

- Bitwise operations / aggregations (#18994)
- Allow `insert_column` to take expressions (#19024)
- Improved error message DSL -> IR resolving (#19032)
- Add `strict` param to eager/lazy frame "rename" (#19017)
- Support `schema` arg in `read/scan_parquet()` (#19013)
- Add `include_file_paths` parameter to `read_parquet` (#19008)
- Add `allow_missing_columns` option to `read/scan_parquet` (#18922)
- Drop python 3.8 support (#18965)
- Use FFI to extract Series from different Polars binaries (#18964)
- Allow for zero-width fixed size lists (#18940)

## 🐞 Bug fixes

- Remove failing temporal lit tests (#19056)
- Divide-by-zero in OOC sort (#19048)
- Ensure `must_flush` flag is not reset (#19046)
- Error node should be on top (#19045)
- Force nested struct `missing` equality (#19031)
- Fix invalid alias udf (#19021)
- Raise invalid predicate join\_where (#19020)
- Fix nested flag of functions with multiple arguments (#19016)
- Fix projection pushdown bug in IEJOINS (#19015)
- Separate temporal tests (#19012)
- Return the truth values of `ne_missing` and `eq_missing` operations for struct instead of `null` (#18930)
- Fix list to numpy conversion (#19009)
- Fix struct broadcasting comparisons (#19003)
- Wrong result on `when().then().otherwise()` on struct when both result are broadcast (#19000)
- Improve literals for temporal subclasses (#18998)
- Ensure same fmt in Series/AnyValue to string cast (#18982)
- Return correct value for `when().then().else()` on structs when using `first()`\\`last()` (#18969)
- IPC don't write variadic\_buffer\_counts in blocks, but only dictionaries (#18980)
- Respect allow\_threading in TernaryExpr (#18977)
- Make join test order-agnostic (#18975)
- Fix `lit().shrink_dtype()` broadcasting (#18958)
- Parallel evaluation of `cumulative_eval` (#18959)
- Properly implement AnyValue::Binary `into_py` (#18960)
- Fix `Expr.over` with `order_by` did not take effect if group keys were sorted (#18947)
- Properly fetch type of full None List Series (#18916)
- Incorrect mode for sorted input (#18945)
- Properly choose inner physical type for Array (#18942)
- Disable very old date in timezone test for CI (#18935)
- Infer reshape dims when determining schema (#18923)
- Incorrect broadcasting on list-of-string set ops (#18918)
- Adding `with_row_index()` to previously collected lazy scan does not take effect (#18913)

## 📖 Documentation

- Fix example of lazy schema verification (#19059)
- Rewrite 'Getting started' page (#19028)
- Fix `is_not_nan` description (#18985)
- Recommend targetDir for rust-analyzer (#18973)
- Fix LazyFrame fetch method references (#18033)

## 📦 Build system

- Bump Rust toolchain to `nightly-2024-09-29` (#19006)
- Bump `simd-json` to 0.14 (#18999)

## 🛠️ Other improvements

- Remove built info (#19057)
- Mark `schema` arg in `read/scan_parquet` as unstable (#19018)
- Fix new-streaming `test_lazy_parquet::test_row_index` (#19019)
- Preserve scalar in more places (#18898)
- Mention `allow_missing_columns` in error message when column not found (parquet) (#18972)
- Disable CSE-specific test on new streaming engine (#18971)
- Add FixedSizeList equality broadcasting (#18967)
- Divide `ChunkCompare` into `Eq` and `Ineq` variants (#18963)
- Another set of new-stream test skip/fixes (#18952)
- Fix/skip variety of new-streaming tests, cont (#18928)
- Fix/skip variety of new-streaming tests (#18924)

Thank you to all our contributors for making this release possible!
@LukasFolwarczny, @Plutone11011, @aleexharris, @alexander-beedie, @barak1412, @coastalwhite, @dependabot, @dependabot[bot], @edwinvehmaanpera, @kgv, @mcrumiller, @nameexhaustion, @orlp, @ritchie46, @rodrigogiraoserrao, @stinodego and @xhiroga