Starstruk Logo MarkStarstruk

v0.20.0

pathwaycom/pathwayv0.20.0Feb 25, 2025by github-actions[bot]

AI Summary

Enhanced DoclingParser with structured chunking and updated column expressions, while changing the image parsing strategy argument.

Key Highlights

  • Added structure-aware chunking for DoclingParser.
  • Column expressions now support `unwrap` and `default` arguments.
  • Changed `parse_images` argument to `image_parsing_strategy`.

Breaking Changes

  • Changed argument in DoclingParser from `parse_images` to `image_parsing_strategy`.
  • doc_post_processors argument no longer accepts pw.UDF.

New Features

  • Structure-aware chunking for DoclingParser.
  • Enhanced column expression arguments.

Full Release Notes

## [0.20.0] - 2025-02-25

### Added
- Added structure-aware chunking for `DoclingParser`.
- Added `table_parsing_strategy` for `DoclingParser`.
- Column expressions `as_int()`, `as_float()`, `as_str()`, and `as_bool()` now accept additional arguments, `unwrap` and `default`, to simplify null handling.
- Support for python tuples in expressions.

### Changed
- **BREAKING**: Changed the argument in `DoclingParser` from `parse_images` (bool) into `image_parsing_strategy` (Literal["llm"] | None).
- **BREAKING**: `doc_post_processors` argument in the `pw.xpacks.llm.document_store.DocumentStore` now longer accepts `pw.UDF`.
- Better error messages when using `pathway spawn` with multiple workers. Now error messages are printed only from the worker experiencing the error directly.

### Fixed
- `doc_post_processors` argument in the `pw.xpacks.llm.document_store.DocumentStore` had no effect. This is now fixed.