rs-0.55.1

This Releases' DSL matches Python 1.43.2.

## 🚀 Performance improvements

- Pushdown `len()` to concat/union inputs (#28570)
- Optimize not(bool\_f) to not\_bool\_f (#28474)
- Split multiplexers that directly scan from in-memory DataFrame (#28376)
- Make DNS cache global (#28352)
- Do not remove cache if predicates not pushed to all inputs (#28341)
- Environment variable for logging slow DNS lookup (#28211)
- Pre-partition on left, right and semi joins on hive partitioned data (#28374)
- Rewrite inner joins on hive partitioned data to a union of partitions filtered joins (#28327)
- Tighten redundant comparisons and propagate filter constraints (#27999)
- Canonicalize predicates (#28212)
- Remove excess memory copy in streaming IPC sink (#28141)
- Parallelize scan\_iceberg resolve (#28168)
- Chunk-aware morsel splitting on single-file IO sink (#27950)
- Don't allocate for `full_null()` with small lengths (#28181)
- Use monotonic deque for rolling min\_by/max\_by (O(n)) (#27436)
- Add a sampled resolve mode for multi-file parquet metadata (#28111)
- Speed up small dtype series sums with a upcast sum kernel (#27958)
- Don't block on path expansion (#28073)
- Add bytes-based concurrency control for cloud IO (#27924)
- Stop flushing multiplexer on phase change (#28013)
- Prevent empty chunks from triggering copy in rechunk (#27953)
- Improve performance of struct unnest inside list eval on streaming (#27905)

## ✨ Enhancements

- Add `struct.drop()` (#28666)
- Improve error message when CSV name de-duplication fails (#28658)
- Always keep first metadata per source for Parquet (#28661)
- Improve plan-time row estimates for multi-file parquet scans (#28380)
- Serialize and deserialize `SinkTypeIR::Partitioned` (#28112)
- Add `infer_schema_files` parameter to `scan_csv` (#28440)
- Support reading IEEE 754 total order Parquet column order (#27896)
- Expose more Expr nodes for cudf\_polars (pt 2) (#28404)
- Allow callback sinks on cloud (#28458)
- Pre-partition group-by on hive keys  (#28444)
- Add ewm\_sum and ewm\_sum\_by #28151 (#28215)
- Serve stale records from object\_store DNS cache (#28256)
- Expose more expr nodes for cudf-polars (#28117)
- Make partition\_hive an opt flag (#28345)
- When displaying IRExprs, only add parentheses when needed for disambiguation (#28319)
- Show only differing dtypes in `assert_frame_equal` error (#27816)
- Add Clone to SpillFrame/Token (#28302)
- Add Series.degrees/radians and clarify trig functions use radians (#22961)
- Merge sorted with multikey (#28067)
- Add `list` expression which consistently packs elements together into new List type (#27990)
- Construct pyarrow predicates via PyO3 instead of eval str (#27271)
- Optimize joins with redundant equi-join keys (#28017)
- Implement missing datetime options for `CastColumnsPolicy` (#28084)
- Add Expr.cat.to and Expr.cat.physical (#28096)
- Add POLARS\_OOC\_DISK\_BUDGET\_MB (#28045)
- Close ResourceWarning leaks in database tests (#28066)
- Expose `IRFunctionExpr::Fused` in the python node visitor (#27889)
- Deprecate casts from string to temporal dtypes (#28056)
- Add naive out-of-core spilling to Polars (#27998)
- Add experimental strict mode (#28023)
- Optimize non-negative len and null\_count comparisons (#28002)
- Optimize len and null\_count comparisons (#27727)
- Support SQL's implicit `JOIN` syntax (#27890)
- Sample from bivariate distribution for spill context Thompson sampling (#27973)
- Add `Expr.is_sorted` (#26708)
- Add suggestions in wrong api usage or mismatched column names (#27954)
- Add external object\_store support for non-native schemes (#27871)
- Skip Delta files and parquet row groups using per-field struct statistics (#27887)
- Add re-registration of OOC SpillTokens (#27909)
- Improve IR sortedness detection for with\_row\_index and explode (#27875)
- Add `DataFrame.is_sorted()` (#27870)

## 🐞 Bug fixes

- Don't hand IO plugins a predicate they cannot evaluate (#28643)
- Don't hang or crash on `sqrt`/`cbrt`/`pct_change`/`ewm_*` with a Struct column (#28696)
- Raise on overflow in Decimal sum instead of silently wrapping (#28688)
- Preserve nulls when importing Arrow maps (#28680)
- Propagate null `by` column values in `rolling_*_by` (#27367)
- Fix self-referencing `field` in `struct.with_fields` with `over` (#28678)
- Fix Arrow buffer offset for `Utf8` and `Binary` (#28662)
- Clamp group-by slice offset (#28579)
- Ensure categorical `fill_null(strategy="min"/"max")` uses lexical min/max (#28636)
- Release the GIL while resolving the schema in `__arrow_c_stream__` (#28642)
- Fix data corruption on Series from nested Arrow `LargeList` (#28632)
- Properly propagate enum metadata for iceberg column mapping (#28580)
- Correctly write parquet field IDs for enums and categoricals (#28581)
- Fix arrow C interop binview offset (#28623)
- Fix duplicate hive values in hive pre-partition rewrite (#28621)
- Double-counted offset for Arrow export of sliced `Series(Array)` (#28602)
- Struct/List/Array equality handling in sort and join (#28601)
- Use try\_new in StructArray construction in polars-json (#27489)
- Raise ComputeError instead of panicking in repeat\_by when output exceeds IdxSize::MAX (#27892)
- Run type coercion pass on pivot's internally generated group\_by (#27897)
- More careful slice pushdown into joins (#28578)
- Preserve ordering in sliced unions (#28576)
- Drop input sortedness when casting to a string (#28574)
- Clear sortedness flags in `StringChunked` substring kernels (#28573)
- Bad mask handling when reading optional parquet column (#28547)
- Any operation on `Unknown(Int)` and `Unknown(Float)` should result in `Unknown(Float)` (#28545)
- Flip `nulls_last` after `Expr.reverse()` (#28572)
- Fix high blocking thread use in sink\_parquet with async local path (#28543)
- Serialize LazyFrames backed by bytes (#28568)
- Release GIL in `SQLContext.execute()` (#28549)
- Propagate `nulls_last` in `function_expr_sortedness` (#28544)
- Incorrect slicing when a join requires sorting (#28541)
- Fix panic on self join of scan\_delta / scan\_iceberg (#28468)
- Decrement ParkGroup worker count when worker exits (#28492)
- Add missing Send bounds in Connector (#28497)
- Fix UB on first/last\_non\_null with empty chunk (#28495)
- Do not CSE non-column height expr on streaming engine (#28480)
- Propagate adjusted sortedness through Gather (#28493)
- Deduplicate the expression equality checks (#28483)
- Address 3VL issue with SQL `NOT IN` interaction with `NULL` values and joins (#28484)
- Use total comparison in Expr literal values (#28482)
- Ensure SQL `SUM` and `CORR` aggregates return NULL for all-null inputs, add `TOTAL` (#28475)
- Share `null_count_dtype` helper between Delta and Iceberg, fixing `SchemaError` (#28479)
- Remove non-output columns from the equi-join and semi/anti-join operators (#28446)
- Fix dropped slice on multiple unions (#28477)
- Fix in-memory engine incorrect slice on maintain order join (#28478)
- Check join schema by position (#28455)
- Block predicate pushdown past overwritten window keys (#28429)
- Avoid panic when union slice skips all rows (#28420)
- Solve panic in `dt.replace` when there were multiple chunks (#28437)
- Propagate `is_scalar` from the input to the output of `.sort()` and `.sort_by()` (#28438)
- Insert missing coercions from `Unknown(_)` in list/array arithmetic (#28411)
- Resolve CSV column names overwrite in DSL->IR conversion (#28383)
- Avoid IEJoin rewrite for Categorical comparisons (#28427)
- Do not rewrite `sort().reverse()` to `sort(descending=True)` when `maintain_order=True` (#28403)
- Propagate bigidx to `polars-plan` from `polars` (#28396)
- Ensure BinaryView offset+len does not exceed i32::MAX where possible (#28048)
- Invalid offset in strptime (#28388)
- Incorrect schema type for decimal \<-> primitive division (#28373)
- Panic in in-memory CSEE handling (#28371)
- Apply same type coercion to IsBetween as binary comparisons (#28300)
- Fix offset in arrow ffi export of sliced struct arrays (#28369)
- Fix cross filter not applied with sink and CSE (#28297)
- Fix panic on projection pushdown with caches (#28280)
- Fix `write_json()` null values in `Array` columns being written incorrectly as `null` (#28330)
- Fix EntityTooSmall on sink\_ipc to S3 (#28255)
- Raise error instead of silent wrapping for `select(len())` (#28355)
- Float16 groupby aggregates (#28361)
- Respect lexical ordering of Categorical in `top_k`/`bottom_k` (#28359)
- Support dtype-list schema overrides in scan\_csv (#28226)
- Resolve unknown type in fused multiply add (#28321)
- Honor `aws_checksum_algorithm` when provided (#28328)
- Fix regression in sourcing HF\_TOKEN (#28317)
- Raise on addition/subtraction between temporal and non-temporal Series (#28266)
- Rewrite with\_columns on empty df to select during DSL -> IR (#28298)
- Fix sink deadlock regression (#28296)
- Avoid panic when consumer of vertical concat is done (#28272)
- Ensure predicate simplification is deterministically ordered (#28269)
- Don't attribute untimed execution to optimization in profile (#28273)
- Harden IPC mmap API (#28258)
- Classify S3 region request failures as IO errors (#28225)
- Incorrect inner dtype in ArrayChunked::from\_aligned\_values (#28249)
- Unsoundness in arg\_sort\_ascending (#28245)
- Add column length check to row encoding (#28248)
- Raise on implode of `Object` dtype instead of creating invalid `List(Object)` (#28246)
- Don't panic in qcut on degenerate float input (all-NaN, infinite, include\_breaks) (#28220)
- Fix streaming Strptime feature flags (#28242)
- Return null not 0 for the first unbiased `ewm_var`/`ewm_std` value (#28235)
- Use after free in file cache (#28236)
- Incorrect length calculation in parallel hashmap (#28227)
- Avoid slice pushdown panic on shared cache inputs (#28157)
- Incorrect dtype for Unknown right literal in arithmetic (#28184)
- Remove unnecessary unsafe Python object downcast (#28189)
- Use-after-free in numpy ufunc FFI (#28195)
- Use-after-free in plugin registry (#28194)
- Unpickling for pl.when().then() (#28165)
- Reduce unknown casts (#28145)
- Consider a `rolling_rank` position invalid if `ost.len() < min_samples` (#28142)
- Incorrect broadcast with empty len in pl.concat\_list (#28134)
- Incorrect sorted fast-path returning nan for grouped max (#28129)
- Fix panic on scan\_parquet filter of fixed-size binary column (#28122)
- Fix decimal dynamic float cast (#28126)
- Fix pyiceberg key dot HDFS prefix filter (#28109)
- Fix projection pushdown panic on `select(len())` after `groupby` (#28108)
- Close ResourceWarning leaks in database tests (#28066)
- Bad array len in UnionArray (#28079)
- Avoid panic on temporal extraction for datetime columns with nulls (#28054)
- Prevent panic in `replace` when `old`/`new` contain Expr or object dtype values (#27433)
- Fix qcut panic and incorrect bins on NaN input (#28020)
- Remove TrustedLen from Scan (#28055)
- Add a deprecation warning for `.explode()` without `empty_as_null` argument (#28040)
- Fix `select(len())` incorrectly returned 0 when using `scan_iceberg` with `pyiceberg` as reader override (#28044)
- Ensure slice expr always observes input order (#28041)
- Fix projection pushdown with row index out of order (#28043)
- Fix cross-join predicates not applied when using `sink_*` functions (#28042)
- Grammar corrections (recommend to → recommend, allows to → allows using) (#28038)
- Ensure optimisations aren't omitted by cached IR nodes (#28004)
- Computed GROUP keys in SQL projections (#27989)
- Handle empty DataFrames in `dt.truncate` and `dt.round` (#26120)
- Address join alias state leakage in SQL CTEs (#27985)
- Support scanning `file://` URIs with percent-encoded paths (#27876)
- Lower SQL `[NOT] IN (subquery)` to semi/anti join (#27888)
- Check for continuity across morsels in AsOf join node (#27461)
- Add test for projection pushdown with hconcat regression (#27937)
- Traverse caches only once in explain (#27939)
- Preserve null placement when collapsing `sort.reverse()` into single sort (#27918)
- Incorrect schema order for hstack in projection pushdown optimizer (#27920)
- Fix duplicated map\_elements warning on streaming engine (#27919)

## 📖 Documentation

- Update and restructure README (#28490)
- Minor fixes to Spark migration guide (#27730)
- Relocate Polars Cloud \& On-Prem User Guide (#28462)
- Add notes on k8s operator (#28445)
- Update comparison page (#28418)
- Update GPU support documentation with the cudf-polars 26.06 release (#27830)
- Fix dev docs being canonical for SEO (#28306)
- Document experimental HDFS support on-premises (#28115)
- Add Series.degrees/radians and clarify trig functions use radians (#22961)
- Add polars-bio to community plugins (#28103)
- Add AWS Marketplace walkthrough to Cloud User Guide (#28214)
- Update config reference (#28150)
- Correct that `merge_sorted()` null inputs should be nulls first (#28177)
- Clarify parts of the Polars Cloud and On-prem user guide (#28170)
- Add On-Premise release 0.6.3 to releases page (#28175)
- Update PolarsCloud snippets for 0.9.0 release  (#28166)
- Fix to\_titlecase incorrectly claiming alphanumeric separation (#28136)
- Fix dataset regression in the examples (#28131)
- Emphasize to generally not ever write Rust tests (#28082)
- Mention Polars skill in User Guide's LLM section (#28113)
- Add StringNameSpaceImpl contains example (#27713)
- Add note on OOM behavior on Kubernetes (#27972)
- Add entry for `DataFrame.is_sorted()` (#27913)
- Add robots.txt disallowing stale docs and added sitemap for better SEO (#27901)
- Update Polars \<> Spark comparison (#27873)

## 📦 Build system

- Propagate `bigidx` to `polars-plan` indirectly included via `pyo3-polars` (#28406)
- Propagate bigidx to `polars-plan` from `polars` (#28396)
- Set the default `dev` profile to `line-tables-only` (#28358)
- Reduce binary size by optimizing for size in the SQL transpilation (#28336)
- Fix test compilation for polars-stream (#27951)

## 🛠️ Other improvements

- Add (with\_)context to PolarsResult for easier annotation (#28677)
- Clean up polars-utils import structure (#28668)
- Deprecate `struct.rename_fields()` with an incorrect number of fields (#28672)
- Re-enable `test_extension()` for streaming engine (#28611)
- Clean up vec utils (#28667)
- Remove Rust compiler intrinsics, use nightly functions (#28665)
- Update `test_hive_join_rewrite_semi_join` test to work with streaming engine (#28610)
- Fix duckdb delta extension install collision (#28607)
- Deprecate casts from `Categorical` to integer dtypes (#28525)
- Deprecate not setting the `plan_stage` argument in `show_graph()` (#28391)
- Make hive\_part extraction a function and public (#28507)
- Deduplicate the expression equality checks (#28483)
- Assert parameter in `filter_scan_ir` (#28472)
- Make the fmt macro exportable (#28459)
- Fix flaky test (#28454)
- Update AI policy for comments (#28436)
- Fix flaky ordering expectation in top\_k\_by (#28435)
- Re-work `shuffle` parameter for `sample()` (#27460)
- SpillFrame instead of DataFrame in Morsel (#28270)
- Deprecate casting numeric types to categoricals (#28349)
- Deprecate `cat.get_categories()` and `cat.to_local()` (#28299)
- Update `spin` dependency to `v0.10.1` (#28360)
- Bloat dispatch reduction (#28337)
- Show only differing dtypes in `assert_frame_equal` error (#27816)
- Accept IR in `optimize` (#28318)
- Fix 1.97 Rust clippy hints (#28305)
- Disallow HashMap/Set in polars-plan (#28276)
- Remove unencountered deny entries (#28260)
- Bump crossbeam-epoch dependency (#28259)
- Deprecate casts from non-nested dtypes into Lists (#28193)
- Deprecate bitwise ops between integers and booleans (#28186)
- Change `object_store` custom DNS resolver patch to rev-based (#28241)
- Remove PartitionedHashMap (#28244)
- Construct pyarrow predicates via PyO3 instead of eval str (#27271)
- Store current SpillContext inside SpillToken (#28076)
- Add `serde(default)` to new cast options DSL (#28209)
- Clarify AI policy, explicitly forbid agents from interacting with our repository (#28206)
- Tighten first-time-contributor screenshot requirements (#28207)
- Ignore quickxml DoS rustsec (#28185)
- Update `typos` to `v1.48.0` (#28162)
- Impl Eq for Scalar (#28144)
- Disallow usage of `tokio::spawn` in clippy (#28123)
- Remove legacy prelude glob imports from leaf files (#28114)
- Set row count estimates for scan\_iceberg (#28075)
- Fix `test_select_explode_height_filter_order_by` failure when `POLARS_MAX_THREADS=2` (#28090)
- Add Morsel::height (#28060)
- Add AI agent disclsoure (#28064)
- Deprecate casts from string to temporal dtypes (#28056)
- Bump memmap2 version (#28050)
- Update `sqlparser` dependency (#28021)
- Update `pyo3` and `rust-numpy` to `0.29.0` (#27970)
- Rename `agg_state.original_len` to `original_groups`, ensure `agg_state.groups` only updated through `with_groups()` (#27747)
- Add `2.0` branch as primary branch (#27978)
- Patch jemalloc-sys for Makefile bug (#27945)
- Restore Cargo.lock (#27964)
- Add test for projection pushdown with hconcat regression (#27937)
- Update `deny.toml` configuration (#27949)
- Doctest skip refactor and add hugging face rate limited tests (#27885)
- Bump Rust dependencies (#27874)

Thank you to all our contributors for making this release possible!
@0guban0v, @AnirudhRahul, @April-Sonnet, @Bharath-970, @BitWeaverDev, @CodingSelim, @EndPositive, @JakubValtar, @Jesse-Bakker, @Kevin-Patyk, @LarryHu0217, @Liyixin95, @MarcoGorelli, @Matt711, @MatteoPossamai, @NicoOhR, @Samoilov2004, @TNieuwdorp, @alex-h-sun, @alexander-beedie, @azimafroozeh, @borchero, @cBournhonesque, @carnarez, @dancsi, @dependabot[bot], @dsprenkels, @gautamvarmadatla, @harrywhalen, @jonasdedden, @jorenham, @kdn36, @kinoko-001, @lzcmian, @makinzm, @malhotrashivam, @masumi-ryugo, @mdavis-xyz, @mdroogh, @mikhail5555, @mkzung, @mroeschke, @nameexhaustion, @nchammas, @orlp, @r-brink, @raphaelroshan, @ritchie46, @subotac, @tylerriccio33, @useredsa, @uurl, @wence-, @xixixao, @yannbolliger, @zichen0116 and [dependabot[bot]](https://github.com/apps/dependabot)