Starstruk Logo MarkStarstruk

v0.16.3

pathwaycom/pathwayv0.16.3Jan 2, 2025by github-actions[bot]

AI Summary

Adds write support for Apache Iceberg and optimizes performance for append-only tables and temporal operators.

Key Highlights

  • Added `pw.io.iceberg.write` method.
  • Values of non-deterministic UDFs are no longer stored in append-only tables.
  • Improved runtime error message for `pw.Table.ix`.

New Features

  • `pw.io.iceberg.write` method.

Full Release Notes

### Added
- `pw.io.iceberg.write` method for writing Pathway tables into Apache Iceberg.

### Changed
- values of non-deterministic UDFs are not stored in tables that are `append_only`.
- `pw.Table.ix` has better runtime error message that includes id of the missing row.

### Fixed
- temporal behaviors in temporal operators (`windowby`, `interval_join`) now consume no CPU when no data passes through them.