v5.5.0
neuml/txtaiv5.5.0Apr 20, 2023by davidmezzetti
AI Summary
Adds workflow streams for server-side processing of large datasets and introduces DuckDB as a new database backend.
Key Highlights
- Workflow streams enable iterative processing of large datasets
- DuckDB backend speeds up non-vector driven queries and aggregations
New Features
- Workflow streams
- DuckDB support
- Translation pipeline langdetect parameter updates
- Pass generation keyword arguments to pipelines
- Replace original prompt in generation pipeline
Full Release Notes
**This release adds workflow streams and DuckDB as a database backend** ↪️️ Workflow streams enable server-side processing of large datasets. Streams iteratively pass content to workflows, no need to pass bulk data through the API. 🦆 DuckDB is a new database backend. Certain larger non-vector driven queries and aggregations will now run significantly faster than with SQLite. See below for full details on the new features, improvements and bug fixes. New Features -------------------------- - Add workflow streams (#461) - Add DuckDB support (#462) Improvements -------------------------- - Modify translation pipeline langdetect parameter to accept language detection function good first issue - Thank you @saucam! (#423, #444) - Pass generation keyword arguments to underlying text generation pipeline (#457) - Replace original prompt in text generation pipeline (#459) Bug Fixes -------------------------- - Issue with upsert and graph (#421) - Upsert API fails with graph config while performing after /delete (#435) - Build errors with latest onnxmltools package (#449) - Fix issue with embeddings reindex and stale function references (#453) - Problem with the workflow builder (#454) - Check for empty queue before attempting to convert inputs to dictionaries (#456) - Fix issue with latest version of Transformers and TokenDetection.save_pretrained (#458)