Starstruk Logo MarkStarstruk

v0.26.2

pathwaycom/pathwayv0.26.2Oct 1, 2025by github-actions[bot]

AI Summary

This release adds Kinesis connectors and SQLite metrics export, fixes memory consumption bugs, and improves performance for Google Drive and background compression operations.

Key Highlights

  • New Kinesis read/write connectors
  • Detailed metrics export to SQLite via PATHWAY_DETAILED_METRICS_DIR
  • GDrive now supports only_metadata format
  • Fixed memory consumption bug in Table.forget and Table.sort during multi-worker runs
  • Improved memory efficiency during cold starts

New Features

  • pw.io.gdrive.read only_metadata format
  • SQLite metrics export via PATHWAY_DETAILED_METRICS_DIR or pw.set_monitoring_config()
  • pw.io.kinesis.read and pw.io.kinesis.write connectors
  • Memory consumption bug fix in Table.forget and Table.sort
  • Improved cold start memory efficiency
  • Background compression frequency optimization for S3/Azure
  • Google Drive connector performance improvements
  • MCP server tool method now supports title, output_schema, annotations, and meta parameters

Full Release Notes

### Added
- `pw.io.gdrive.read` now supports the `"only_metadata"` format. When this format is used, the table will contain only metadata updates for the tracked directory, without reading object contents.
- Detailed metrics can now be exported to SQLite. Enable this feature using the environment variable `PATHWAY_DETAILED_METRICS_DIR` or via `pw.set_monitoring_config()`.
- `pw.io.kinesis.read` and `pw.io.kinesis.write` methods for reading from and writing to AWS Kinesis.

### Fixed
- A bug leading to potentially unbounded memory consumption that could occur in `Table.forget` and `Table.sort` operators during multi-worker runs has been fixed.
- Improved memory efficiency during cold starts by compacting intermediary structures and reducing retained memory after backfilling.

### Changed
- The frequency of background operator snapshot compression in data persistence is limited to the greater of the user-defined `snapshot_interval` or 30 minutes when S3 or Azure is used as the backend, in order to avoid frequent calls to potentially expensive operations.
- The Google Drive input connector performance has been improved, especially when handling directories with many nested subdirectories.
- The MCP server `tool` method now allows to pass the optional data `title`, `output_schema`, `annotations` and `meta` to inform the LLM client.
- Relaxed boto3 dependency to <2.0.0.