v0.21.6
pathwaycom/pathwayv0.21.6May 29, 2025by github-actions[bot]
AI Summary
This release adds a sort_by method to BaseCustomAccumulator, improves debug output, deprecates VectorStoreServer and VectorStoreClient in favor of DocumentStore variants, and enhances Delta Lake write capabilities.
Key Highlights
- sort_by method for BaseCustomAccumulator to process rows in specified order
- pw.xpacks.llm.vector_store.VectorStoreServer deprecated in favor of DocumentStore
- pw.xpacks.llm.vector_store.VectorStoreClient deprecated in favor of DocumentStoreClient
- pw.io.deltalake.write can now maintain target table's snapshot
New Features
- sort_by method for BaseCustomAccumulator
- VectorStoreServer and VectorStoreClient deprecated in favor of DocumentStore variants
- pw.io.deltalake.write can maintain target table's snapshot
Full Release Notes
### Added
- `sort_by` method to `pw.BaseCustomAccumulator` that allows to sort rows within a single batch. When `sort_by` is defined the rows are reduced in the order specified by the `sort_by` method. It can for example be used to process entries in the order of event time.
### Changed
- `pw.Table.debug` now prints a whole row in a single line instead of printing each cell separately.
- Calling functions without arguments in YAML configurations files is now deprecated in `pw.load_yaml`. To call the function a mapping should be passed, e.g. empty mapping as `{}`. In the future `!` syntax without any mapping will be used to pass function objects without calling them.
- The license check error message now provides a more detailed explanation of the failure.
- When code is run using `pathway spawn` with multiple processes, if one process terminates with an error, all other processes will also be terminated.
- `pw.xpacks.llm.vector_store.VectorStoreServer` is being deprecated, and it is now subclass of `pw.xpacks.llm.document_store.DocumentStore`. Public API is being kept the same, however users are encouraged to switch to using `DocumentStore` from now on.
- `pw.xpacks.llm.vector_store.VectorStoreClient` is being deprecated in favor of `pw.xpacks.llm.document_store.DocumentStoreClient`.
- `pw.io.deltalake.write` can now maintain the target table's snapshot on the output.