Starstruk Logo MarkStarstruk

v0.26.1

pathwaycom/pathwayv0.26.1Aug 28, 2025by github-actions[bot]

AI Summary

This release adds utility methods for table manipulation, enables async callbacks for certain I/O connectors, and fixes PostgreSQL type optionality handling.

Key Highlights

  • New Table methods: forget, buffer, and ignore_late for time-based data manipulation
  • Async UDF row batching with max_batch_size parameter
  • PostgreSQL diff column now uses smallint type
  • Improved PostgreSQL type handling in initialization modes

New Features

  • pw.Table.forget
  • pw.Table.buffer
  • pw.Table.ignore_late
  • Rows batching for async UDFs with max_batch_size parameter

Full Release Notes

### Added
- `pw.Table.forget` to remove old (in terms of event time) entries from the pipeline.
- `pw.Table.buffer`, a stateful buffering operator that delays entries until `time_column <= max(time_column) - threshold` condition is met.
- `pw.Table.ignore_late` to filter out old (in terms of event time) entries.
- Rows batching for async UDFs. It can be enabled with `max_batch_size` parameter.

### Changed
- `pw.io.subscribe` and `pw.io.python.write` now work with async callbacks.
- The `diff` column in tables automatically created by `pw.io.postgres.write` and `pw.io.postgres.write_snapshot` in `replace` and `create_if_not_exists` initialization modes now uses the `smallint` type.
- `optimize_transaction_log` option has been removed from `pw.io.deltalake.TableOptimizer`.

### Fixed
- `pw.io.postgres.write` and `pw.io.postgres.write_snapshot` now respect the type optionality defined in the Pathway table schema when creating a new PostgreSQL table. This applies to the `replace` and `create_if_not_exists` initialization modes.