v0.4.0
microsoft/graphragv0.4.0Nov 6, 2024by AlonsoGuevara
AI Summary
Introduces Incremental Indexing and the DRIFT graph reasoning module, alongside significant refactoring of workflows, embeddings, and the CLI. Includes performance optimizations and runtime-only storage options.
Key Highlights
- Added Incremental Indexing support
- Introduced DRIFT graph reasoning query module
- Refactored embeddings into a different workflow
- Added DRIFT search CLI and example notebook
- Reorganized API, reporter, and callback code into separate components
New Features
- Incremental Indexing
- DRIFT graph reasoning query module
- DRIFT search CLI and example notebook
- Runtime-only storage option
- Naive community merge using time period
Full Release Notes
## What's Changed - minor: Add Incremental Indexing - minor: Added DRIFT graph reasoning query module - minor: embeddings moved to a different workflow - minor: Add DRIFT search cli and example notebook - patch: Add config for incremental updates - patch: Add embeddings to subflow. - patch: Add naive community merge using time period - patch: Add relationship merge - patch: Add runtime-only storage option. - patch: Add text units update - patch: Allow empty workflow returns to avoid disk writing. - patch: Apply pandas optimizations to create final entities - patch: Calculate new inputs and deleted inputs on update - patch: Collapse covariates flow. - patch: Collapse create-base-entity-graph. - patch: Collapse create-final-community-reports. - patch: Collapse create-final-documents. - patch: Collapse create-final-entities. - patch: Collapse create-final-nodes. - patch: Collapse create_base_documents. - patch: Collapse create_base_text_units. - patch: Collapse create_final_relationships. - patch: Collapse entity extraction. - patch: Collapse entity summarize. - patch: Collapse intermediate workflow outputs. - patch: Dependency updates - patch: Extract DataShaper-less flows. - patch: Fix Community ID loading for DRIFT search over existing indexes - patch: Fix embeddings faulty assignments - patch: Fix init defaults for vector store and drift img in docs - patch: Fix nested json parsing - patch: Fix some edge cases on Drift Search over small input sets - patch: Fix var name for embedding - patch: Merge existing and new entities, updating values accordingly - patch: Merge text_embed into create-final-relationships subflow. - patch: Move embedding verbs to operations. - patch: Moving verbs around. - patch: Optimize Create Base Documents subflow - patch: Optimize text unit relationship count - patch: Perf optimizations in map_query_to_entities() - patch: Remove aggregate_df from final coomunities and final text units - patch: Remove duplicated relationships and nodes - patch: Remove unused column from final entities - patch: Reorganized api,reporter,callback code into separate components. Defined debug profiles. - patch: Small cleanup in community context history building - patch: Transient entity graph and snapshotting. - patch: Update Incremental Indexing to new embeddings workflow - patch: Use mkdocs for documentation - patch: add backwards compatibility patch to vector store. - patch: add-autogenerated-cli-docs - patch: fix docs image path - patch: refactor use of vector stores and update support for managed identity - patch: remove redundant error-handling code from global-search - patch: reorganize cli layer **Full Changelog**: https://github.com/microsoft/graphrag/compare/v0.3.6...v0.4.0