py-1.9.0
pola-rs/polarspy-1.9.0Oct 1, 2024by github-actions[bot]
AI Summary
Python release focusing on performance improvements, bitwise operations, and new Parquet options.
Key Highlights
- Use List's TotalEqKernel
- Bitwise operations / aggregations
- Support `schema` arg in `read/scan_parquet()`
New Features
- Bitwise operations / aggregations
- Allow `insert_column` to take expressions
- Improved error message DSL -> IR resolving
- Add `strict` param to eager/lazy frame "rename"
- Support `schema` arg in `read/scan_parquet()`
- Add `include_file_paths` parameter to `read_parquet`
- Add `allow_missing_columns` option to `read/scan_parquet`
- Drop python 3.8 support
- Use FFI to extract Series from different Polars binaries
- Allow for zero-width fixed size lists
Full Release Notes
## 🚀 Performance improvements - Use List's TotalEqKernel (#18984) ## ✨ Enhancements - Bitwise operations / aggregations (#18994) - Allow `insert_column` to take expressions (#19024) - Improved error message DSL -> IR resolving (#19032) - Add `strict` param to eager/lazy frame "rename" (#19017) - Support `schema` arg in `read/scan_parquet()` (#19013) - Add `include_file_paths` parameter to `read_parquet` (#19008) - Add `allow_missing_columns` option to `read/scan_parquet` (#18922) - Drop python 3.8 support (#18965) - Use FFI to extract Series from different Polars binaries (#18964) - Allow for zero-width fixed size lists (#18940) ## 🐞 Bug fixes - Remove failing temporal lit tests (#19056) - Divide-by-zero in OOC sort (#19048) - Ensure `must_flush` flag is not reset (#19046) - Error node should be on top (#19045) - Force nested struct `missing` equality (#19031) - Fix invalid alias udf (#19021) - Raise invalid predicate join\_where (#19020) - Fix nested flag of functions with multiple arguments (#19016) - Fix projection pushdown bug in IEJOINS (#19015) - Separate temporal tests (#19012) - Return the truth values of `ne_missing` and `eq_missing` operations for struct instead of `null` (#18930) - Fix list to numpy conversion (#19009) - Fix struct broadcasting comparisons (#19003) - Wrong result on `when().then().otherwise()` on struct when both result are broadcast (#19000) - Improve literals for temporal subclasses (#18998) - Ensure same fmt in Series/AnyValue to string cast (#18982) - Return correct value for `when().then().else()` on structs when using `first()`\\`last()` (#18969) - IPC don't write variadic\_buffer\_counts in blocks, but only dictionaries (#18980) - Respect allow\_threading in TernaryExpr (#18977) - Make join test order-agnostic (#18975) - Fix `lit().shrink_dtype()` broadcasting (#18958) - Parallel evaluation of `cumulative_eval` (#18959) - Properly implement AnyValue::Binary `into_py` (#18960) - Fix `Expr.over` with `order_by` did not take effect if group keys were sorted (#18947) - Properly fetch type of full None List Series (#18916) - Incorrect mode for sorted input (#18945) - Properly choose inner physical type for Array (#18942) - Disable very old date in timezone test for CI (#18935) - Infer reshape dims when determining schema (#18923) - Incorrect broadcasting on list-of-string set ops (#18918) - Adding `with_row_index()` to previously collected lazy scan does not take effect (#18913) ## 📖 Documentation - Fix example of lazy schema verification (#19059) - Rewrite 'Getting started' page (#19028) - Fix `is_not_nan` description (#18985) - Recommend targetDir for rust-analyzer (#18973) - Fix LazyFrame fetch method references (#18033) ## 📦 Build system - Bump Rust toolchain to `nightly-2024-09-29` (#19006) - Bump `simd-json` to 0.14 (#18999) ## 🛠️ Other improvements - Remove built info (#19057) - Mark `schema` arg in `read/scan_parquet` as unstable (#19018) - Fix new-streaming `test_lazy_parquet::test_row_index` (#19019) - Preserve scalar in more places (#18898) - Mention `allow_missing_columns` in error message when column not found (parquet) (#18972) - Disable CSE-specific test on new streaming engine (#18971) - Add FixedSizeList equality broadcasting (#18967) - Divide `ChunkCompare` into `Eq` and `Ineq` variants (#18963) - Another set of new-stream test skip/fixes (#18952) - Fix/skip variety of new-streaming tests, cont (#18928) - Fix/skip variety of new-streaming tests (#18924) Thank you to all our contributors for making this release possible! @LukasFolwarczny, @Plutone11011, @aleexharris, @alexander-beedie, @barak1412, @coastalwhite, @dependabot, @dependabot[bot], @edwinvehmaanpera, @kgv, @mcrumiller, @nameexhaustion, @orlp, @ritchie46, @rodrigogiraoserrao, @stinodego and @xhiroga