py-1.23.0
pola-rs/polarspy-1.23.0Feb 23, 2025by github-actions[bot]
AI Summary
Adds SQL support for the `DELETE` statement and implements `merge_sorted` for struct and binary types.
Key Highlights
- Add SQL support for the `DELETE` statement
- Implement `merge_sorted` for struct
- Add `Int128` path for `join_asof`
New Features
- Implement i128 -> str cast
- Version DSL
- Make user facing binary formats mostly self describing
- Add SQL support for the `DELETE` statement
- Add row index to new streaming multiscan
- Implement `merge_sorted` for struct
- Add `linear_spaces`
Full Release Notes
## 🚀 Performance improvements - Toggle projection pushdown for eager rolling (#21405) - Fix pathologic `rolling + group-by` performance and memory explosion (#21403) - Add sampling to new-streaming equi join to decide between build/probe side (#21197) ## ✨ Enhancements - Implement i128 -> str cast (#21411) - Connect polars-cloud (#21387) - Version DSL (#21383) - Make user facing binary formats mostly self describing (#21380) - Filter hive files using predicates in new streaming (#21372) - Add negative slicing to new streaming multiscan (#21219) - Allow iterable of frames as input to `align_frames` (#21209) - Implement sorted flags for struct series (#21290) - Support reading arrow Map type from Delta (#21330) - Add a dedicated `remove` method for `DataFrame` and `LazyFrame` (#21259) - Rename `credentials` parameter to `credential` in `CredentialProviderAzure` (#21295) - Implement `merge_sorted` for struct (#21205) - Add positive slice for new streaming MultiScan (#21191) - Don't take in rewriting visitor (#21212) - Add SQL support for the `DELETE` statement (#21190) - Add row index to new streaming multiscan (#21169) - Improve DataFrame fmt in explain (#21158) ## 🐞 Bug fixes - Method `dt.ordinal_day` was returning UTC results as opposed to those on the local timestamp (#21410) - Use Kahan summation for rolling sum kernels. Fix numerical stability issues (#21413) - Add scalar checks for `n` and `fill_value` parameters in `shift` (#21292) - Upcast small integer dtypes for rolling sum operations (#21397) - Don't silently produce null values from invalid input to `pl.datetime` and `pl.date` (#21013) - Allow duration multiplied w/ primitive to propagate in IR schema (#21394) - Struct arithmetic broadcasting behavior (#21382) - Prefiltered optional plain primitive kernel (#21381) - Panic when projecting only row index from IPC file (#21361) - Properly update groups after `gather` in aggregation context (#21369) - Mark test as may\_fail\_auto\_streaming (#21373) - Properly set `fast_unique` in EnumBuilder (#21366) - Rust test race condition (#21368) - Fix unequal DataFrame column heights from parquet hive scan with filter (#21340) - Fix ColumnNotFound error selecting `len()` after semi/anti join (#21355) - Merge Parquet nested and flat decoders (#21342) - Incorrect atomic ordering in Connector (#21341) - Method `dt.offset_by` was discarding month and year info if day was included in offset for timezone-aware columns (#21291) - Fix pickling `polars.col` on Python versions \<3.11 (#21333) - Fix duplicate column names after join if suffix already present (#21315) - Skip Batches Expression for boolean literals (#21310) - Fix performance regression for eager `join_where` (#21308) - Fix incorrect predicate pushdown for predicates referring to right-join key columns (#21293) - Panic in `to_physical` for series of arrays and lists (#21289) - Resolve deadlock due to leaking in Connector recv drop (#21296) - Incorrect result for merge\_sorted with lexical categorical (#21278) - Add `Int128` path for `join_asof` (#21282) - Categorical min/max returning String dtype rather than Categorical (#21232) - Checking overflow in Sliced function (#21207) - Adding a struct field using a literal raises InvalidOperationError (#21254) - Return nulls for `is_finite`, `is_infinite`, and `is_nan` when dtype is `pl.Null` (#21253) - Account for minor change in new `connectorx` release (#21277) - Properly implement and test Skip Batch Predicate (#21269) - Infinite recursion when broadcasting into struct zip\_outer\_validity (#21268) - Deadlock due to bad logic in new-streaming join sampling (#21265) - Incorrect result for top\_k/bottom\_k when input is sorted (#21264) - UTF-8 validation of nested string slice in Parquet (#21262) - Raise instead of panicking when casting a Series to a Struct with the wrong number of fields (#21213) - Defer credential provider resolution to take place at query collection instead of construction (#21225) - Do not panic in `strptime()` if `format` ends with '%' (#21176) - Raise error instead of panicking for unsupported SQL operations (#20789) - Projection of only row index in new streaming IPC (#21167) - Fix projection count query optimization (#21162) ## 📖 Documentation - Fix doc for SQL Functions navigation (#21412) - Fix initial selector example (#21321) - Add pandas strictness API difference (#21312) - Improve `Expr.name.map` docstring example (#21309) - Add logo to Ask AI (#21261) - Fix docs for Catalog (#21252) - AI widget again (#21257) - Revert plugin (#21250) - Add kappa ask ai widget (#21243) - Update social icons in API reference docs (#21214) - Improve Arrow key feature description (#21171) - Improve example in IO plugins user guide (#21146) ## 🛠️ Other improvements - Move storage of hive partitions to DataFrame (#21364) - Feature gate merge sorted in new streaming engine (#21338) - Remove new streaming old multiscan (#21300) - Add tests for fixed open issues (#21185) - Try to mimic all steps (#21249) - Require version for POLARS\_VERSION (#21248) - Fix docs (#21246) - Avoid unnecessary `packaging` dependency (#21223) - Remove unused file (#21240) - Add use\_field\_init\_shorthand = true to rustfmt (#21237) - Don't mutate arena by default in Rewriting Visitor (#21234) - Disable the TraceMalloc allocator (#21231) - Add feature gate to old streaming deprecation warning (#21179) - Install seaborn when running remote benchmark (#21168) Thank you to all our contributors for making this release possible! @GiovanniGiacometti, @JakubValtar, @MarcoGorelli, @Matt711, @Shoeboxam, @YichiZhang0613, @alexander-beedie, @bschoenmaeckers, @coastalwhite, @edwinvehmaanpera, @erikbrinkman, @etiennebacher, @hemanth94, @henryharbeck, @jqnatividad, @lukemanley, @mcrumiller, @nameexhaustion, @orlp, @r-brink, @ritchie46 and @ydagosto