py-1.41.0

## 🏆 Highlights

- Add LazyFrame.gather (#27501)
- Nested common subplan elimination (#27340)
- Stabilize streaming engine (#27497)
- Speed up parquet metadata decode with hand-written Thrift (#27427)

## ⚠️ Deprecations

- Deprecate the StringCache (#27580)

## 🚀 Performance improvements

- Dispatch `{list,arr}.{unique,n_unique,reverse}` to group\_by engine (#27278)
- Improve in-memory grouped non-null count (#27702)
- Factor shared conjuncts out of OR-of-ANDs predicates (#27627)
- Skip downloading IPC batches exceeding slice bounds (#27683)
- Avoid materializing broadcast list in `list.shift` (#27628)
- Optimise `json_decode` Datetime string parsing (#27559)
- Speed up `to_numpy` C-order via cache-blocked transpose (#27522)
- Optimize `select(len())` for non-strict horizontal concat (#27516)
- Pushdown slices to inputs on left/right/full join (#27508)
- Don't infer CSV schema if schema is set (#27507)
- Nested common subplan elimination (#27340)
- Make `is_in` row-group pruning precise on null-containing haystacks (#27495)
- Don't do fused-multiply-add on scalars (#27479)
- List full fast path (#27477)
- Make `is_in` row-group pruning precise on multi-value lists (#27475)
- Add streaming GatherNode (#27465)
- Lower non-elementwise FunctionExprIR to ColumnarFunctionNode (#27462)
- Speed up parquet metadata decode with hand-written Thrift (#27427)

## ✨ Enhancements

- Use true division for the `/` operator in Polars SQL (#27391)
- Add Rust backend for Expr.has\_nulls (#27590)
- Stabilize float16 (#27607)
- Add Expr.is\_empty (#27583)
- Add support for the SQL `FILTER` clause for aggregate functions, and `STRING_AGG` (#27564)
- Make parquet `FileMetadata` prunable for IR-plan dispatch (#27535)
- Broadcast scalar input for `list.slice` (#27487)
- Add LazyFrame.gather (#27501)
- Add `null_on_oob` in {Expr/Series}.gather (#27327)
- Stabilize streaming engine (#27497)
- Process batched `arr.eval` on overflow boundaries (#27496)
- Process batched `list.eval` on overflow boundaries (#27483)
- Print `SLICED UNION` in LazyFrame explain (#27467)

## 🐞 Bug fixes

- Panic in scan of empty IPC with slice (#27708)
- Persist object\_store rebuild state in cache (#27707)
- Sort flag on GroupsType only applies to first element (#27684)
- Invalid unwrap\_unchecked when length isn't exact (#27685)
- Don't unwrap channel send in streaming join\_asof (#27688)
- Fix `merge_sorted` panic when List in frame (#27568)
- Put AsOf join buffered Morsels back the front of the deque if we cannot process them rn (#27658)
- Fix skip\_batches logic for NaN (#27673)
- Raise `TypeError` when calling `next()` directly on `GroupBy` objects (#27562)
- Data type comparison for extension types (#27632)
- Share last-morsel split budget across files in streaming multi-scan (#27630)
- Bytes scalars were not being broadcast in dataframe constructor (#27621)
- Reset the sort-options in `Series::is_sorted()` after row-encoding columns (#27614)
- Rayon deadlock with re-entrant io sources (#27600)
- Don't push negative-offset slices through `HConcat` (#27570)
- Logic error in streaming is\_empty (#27602)
- Fix incorrect CSE with large is\_in literal (#27575)
- AnonymousFunction can qualify as SQL aggregator (#26986)
- Fix CSPE panic in cloud (#27594)
- Set merge-join streaming node to `Finished` if its sending port is `Done` (#27572)
- Widen decimal precision on sum aggregation at runtime (#27579)
- Fix `str.to_time` was raising unnecessarily when input was all nulls (#27574)
- Prevent panic when switching from one extension dtype to another (#27566)
- Fix `DataFrame.write_database(..., if_table_exists="append", engine="adbc")` not handling missing tables correctly (#26913)
- Ensure `json_decode` doesn't fail for Date and Time string deserialization (#27554)
- Incorrect RUSTFLAGS passing in Makefile (#27555)
- Fix panic on reading IPC with 0-row compressed bitmap (#27551)
- Set HEAD\_RESPONSE\_SIZE\_ESTIMATE to 0 (#27548)
- Fix lazy concat horizontal didn't raise on mismatching heights after projection pushdown (#27506)
- Prevent join panic when `suffix=""` and `coalesce=True` (#27376)
- Do not make a `FastCount` for csv if `pre_slice` is set (#27536)
- Support duplicate names in `over` (#27544)
- Reassign sequence numbers when distributing input morsels in streaming AsOf join node (#27538)
- Do not reverse dataframes when sorting with all-null key columns (#27517)
- Incorrect length check on streaming zip (#27505)
- Remove invalid type annotation ``Sequence[int]`` from ``DataFrame.\_\_setitem\_\_`` ``key`` (#27355)
- Respect `nulls_last` for descending over(order\_by) in `group_by().agg()` (#27486)
- Fix perf regression in `scan_csv` `select(len())` when collected on streaming engine (#27504)
- Harden extend strictness (#27476)
- Prevent deadlock when using `to_arrow()` in a multithreaded context (#27472)
- Do not flatten sliced union (#27466)
- Prevent deadlock when using `to_pandas()` in multithreaded context (#27451)
- Struct rechunk bug and add Series::with\_validity (#27446)
- Handle column indexing in `read_parquet`/`read_csv` with pyarrow reader (#27397)
- Export enum as ordered dictionary to arrow (#27432)
- Ensure `sample()` respects `shuffle=False` (#27248)
- Return empty `DataFrame` from `concat_list` with `lit` and empty column (#27305)
- Read parquet `MAP` columns without `LogicalType` annotation (#27404)
- Raise `DuplicateError` on parquet files with duplicate column names (#27399)

## 📖 Documentation

- Document Expr.list.\_\_getitem\_\_ (#27689)
- Add cloudpickle requirement (#27703)
- Clarify from\_arrow schema ordering (#27493)
- Fix a typo in `join_asof` docstring (#27682)
- Clarify schema column order (#27681)
- Document horizontal string concatenation (#27542)
- Document all valid `engine` options on LazyFrame collect/sink/explain methods (#27374)
- Orchestration docs check (#27605)
- Drop redundant Pattern 2 from Dagster integration page (#27581)
- Update to remove Dockerhub PAT references (#27582)
- Modernize Dagster integration example for Polars Cloud (#27560)
- Use Polars random seed in sample example (#27537)
- Clarify full join description (#27530)
- Make expressions operations RNG deterministic (#27494)
- Document struct field order (#27492)
- Improve `over:order_by` description (#27520)
- Clarify join output columns (#27449)
- Document null propagation in pl.format (#27447)
- Document gzip support in read\_csv (#27434)
- Add See Also sections for datetime docstrings (#27316)
- Polars On-Prem release (#27439)
- Rename to Polars On-Prem (#27435)
- Clarify null handling in unique operations (#27431)
- Document `write_ipc` buffer behavior with `file=None` (#27430)

## 📦 Build system

- Also split debug info in debug-release (#27609)
- Use split-debuginfo on linux (#27608)
- Bump deltalake to 1.5.1 in CI (#27387)

## 🛠️ Other improvements

- Remove redundant DSL::AGG::Unique (#27718)
- Harden against async blocking deadlocks (#27653)
- Print Python traceback when POLARS\_TIMEOUT\_MS is exceeded (#27657)
- Format missed in previous PR (#27700)
- Bump pytest and remove codspeed (#27686)
- Remove client-side `allow_local_scans` option for `prepare_cloud_plan` (#27663)
- Remove superfluous test (#27676)
- Cleanup streaming flags (#27671)
- Expose unordered concatenation in python visitor (#27666)
- Bump `deltalake` and fix CI (#27660)
- Add `impl IntoAExprBuilder` for `ExprIR` (#27656)
- Split `_expand_selector_dicts` into multiple functions so return type is simple and accurate (#27618)
- Update object\_store patch repo (#27650)
- Match NumPy signature in `DataFrame.__array__` and `Series.__array__` (#27634)
- Add ImageVersion to rust-cache key (#27626)
- Run Pyrefly on tests (#27459)
- Fix is\_empty test (#27597)
- Fix tz type difference pandas assert, take 2 (#27596)
- Fix CSPE panic in cloud (#27594)
- Fix tz type difference pandas assert (#27593)
- Add contributing note about conventional comments (#27543)
- Nested common subplan elimination (#27340)
- Deduplicate interns (#27470)
- Fix merge conflict in ColumnarFunction (#27464)
- Keep the schema ordered in scan projection pushdown (#27429)
- Remove unused `type: ignore` statements (#27360)
- Remove redundant `PhysNodeKind::AsOfJoin::{left_right}_by` fields (#27400)
- Resolve type-ignores in `udfs.py` (#27341)
- Bump rustls-webpki (#27382)

Thank you to all our contributors for making this release possible!
@0guban0v, @EndPositive, @JakubValtar, @Jesse-Bakker, @Kevin-Patyk, @Liyixin95, @MarcoGorelli, @NedJWestern, @Shoeboxam, @SuryaSunil1326, @TNieuwdorp, @alexander-beedie, @aryansri05, @ashler-herrick, @azimafroozeh, @carnarez, @coastalwhite, @dependabot[bot], @dsprenkels, @gab23r, @gautamvarmadatla, @ilya-pevzner, @jonathansergio, @junnythemarksman, @kdn36, @lun3x, @nameexhaustion, @orlp, @pablogsal, @ritchie46, @uurl, @waamm, @wence-, @wmoss, @xronocode and [dependabot[bot]](https://github.com/apps/dependabot)