v0.21.6

pathwaycom/pathwayv0.21.6May 29, 2025by github-actions[bot]

AI Summary

This release adds a sort_by method to BaseCustomAccumulator, improves debug output, deprecates VectorStoreServer and VectorStoreClient in favor of DocumentStore variants, and enhances Delta Lake write capabilities.

Key Highlights

  • sort_by method for BaseCustomAccumulator to process rows in specified order
  • pw.xpacks.llm.vector_store.VectorStoreServer deprecated in favor of DocumentStore
  • pw.xpacks.llm.vector_store.VectorStoreClient deprecated in favor of DocumentStoreClient
  • pw.io.deltalake.write can now maintain target table's snapshot

New Features

  • sort_by method for BaseCustomAccumulator
  • VectorStoreServer and VectorStoreClient deprecated in favor of DocumentStore variants
  • pw.io.deltalake.write can maintain target table's snapshot

Full Release Notes

### Added
- `sort_by` method to `pw.BaseCustomAccumulator` that allows to sort rows within a single batch. When `sort_by` is defined the rows are reduced in the order specified by the `sort_by` method. It can for example be used to process entries in the order of event time.

### Changed
- `pw.Table.debug` now prints a whole row in a single line instead of printing each cell separately.
- Calling functions without arguments in YAML configurations files is now deprecated in `pw.load_yaml`. To call the function a mapping should be passed, e.g. empty mapping as `{}`. In the future `!` syntax without any mapping will be used to pass function objects without calling them.
- The license check error message now provides a more detailed explanation of the failure.
- When code is run using `pathway spawn` with multiple processes, if one process terminates with an error, all other processes will also be terminated.
- `pw.xpacks.llm.vector_store.VectorStoreServer` is being deprecated, and it is now subclass of `pw.xpacks.llm.document_store.DocumentStore`. Public API is being kept the same, however users are encouraged to switch to using `DocumentStore` from now on. 
- `pw.xpacks.llm.vector_store.VectorStoreClient` is being deprecated in favor of `pw.xpacks.llm.document_store.DocumentStoreClient`.
- `pw.io.deltalake.write` can now maintain the target table's snapshot on the output.