Starstruk Logo MarkStarstruk

v0.20.1

pathwaycom/pathwayv0.20.1Mar 7, 2025by github-actions[bot]

AI Summary

Added a RecursiveSplitter utility and implemented schema validation for Delta Lake writes to ensure compatibility.

Key Highlights

  • Added RecursiveSplitter for document processing.
  • Delta Lake write now validates schema against the target table.

New Features

  • RecursiveSplitter utility.
  • Delta Lake schema validation.

Full Release Notes

### Added
- Added `RecursiveSplitter`
- `pw.io.deltalake.write` now checks that the schema of the target table Delta Table corresponds to the schema of the Pathway table that is sent for the output. If the schemas differ, a human-readable error message is produced.