v0.30.1
pathwaycom/pathwayv0.30.1Apr 23, 2026by github-actions[bot]
AI Summary
This release introduces multiple new database and messaging connectors including RabbitMQ, MSSQL, Milvus, and LEANN, while also enhancing multi-machine deployment capabilities with new spawn flags and adding operator persistence for iterate operations.
Key Highlights
- New RabbitMQ connectors (read/write) supporting JSON, plaintext, raw formats with streaming/static modes and TLS
- New MSSQL connectors (read/write) with CDC support for change tracking
- New Milvus write connector for vector storage with upsert/delete operations
- pathway spawn now supports --addresses and --process-id flags for TCP-based multi-machine clusters
- New AudioParser using OpenAI Whisper API for audio transcription
New Features
- pw.io.rabbitmq.read and pw.io.rabbitmq.write connectors
- pw.io.mssql.read connector with CDC support
- pw.io.mssql.write connector with MERGE/DELETE operations
- pw.io.milvus.write connector
- pathway spawn --addresses and --process-id flags for multi-machine deployments
- pw.xpacks.llm.parsers.AudioParser based on OpenAI Whisper API
- pw.io.leann.write connector for LEANN vector indices
- pw.iterate operator persistence for incremental processing
Full Release Notes
### Added - `pw.io.rabbitmq.read` and `pw.io.rabbitmq.write` connectors for reading from and writing to RabbitMQ Streams. Supports JSON, plaintext, and raw formats; streaming and static modes; persistence with offset recovery; dynamic topics (writing to different streams per row); `start_from` parameter (`"beginning"`, `"end"`, or `"timestamp"`); TLS configuration; and message metadata including AMQP 1.0 properties and application properties. Header values are JSON-encoded for round-trip compatibility. Requires a Pathway Scale or Enterprise license. - `pw.io.mssql.read` connector, which reads data from a Microsoft SQL Server table. The connector first delivers a full snapshot of the table and then, if the streaming mode is used, tracks incremental changes via SQL Server Change Data Capture (CDC). - `pw.io.mssql.write` connector, which writes a Pathway table to a Microsoft SQL Server table. Row additions and updates are applied as MERGE (upsert) statements keyed on the configured primary key columns, and row deletions are applied as DELETE statements. - `pw.io.milvus.write` connector, which writes a Pathway table to a Milvus collection. Row additions are sent as upserts and row deletions are sent as deletes keyed on the configured primary key column. Requires a Pathway Scale license. - `pathway spawn` now supports the `--addresses` and `--process-id` flags for multi-machine deployments. Pass a comma-separated list of `host:port` addresses for all processes and the index of the local process; Pathway will connect the cluster over TCP without requiring all processes to run on the same machine. - `pw.xpacks.llm.parsers.AudioParser`, audio transcription parser based on OpenAI Whisper API. Accepts raw audio bytes and returns transcribed text, following the same interface as other Pathway document parsers. - `pw.io.leann.write` connector for writing Pathway tables to LEANN vector indices. LEANN uses graph-based selective recomputation to achieve 97% storage reduction compared to traditional vector databases. - `pw.iterate` now supports operator persistence. On restart, the iterate operator loads its previous input from an operator snapshot and reconverges inside the loop, allowing incremental processing of new data without replaying the full input stream.