v0.4.0

microsoft/graphragv0.4.0Nov 6, 2024by AlonsoGuevara

AI Summary

Introduces Incremental Indexing and the DRIFT graph reasoning module, alongside significant refactoring of workflows, embeddings, and the CLI. Includes performance optimizations and runtime-only storage options.

Key Highlights

  • Added Incremental Indexing support
  • Introduced DRIFT graph reasoning query module
  • Refactored embeddings into a different workflow
  • Added DRIFT search CLI and example notebook
  • Reorganized API, reporter, and callback code into separate components

New Features

  • Incremental Indexing
  • DRIFT graph reasoning query module
  • DRIFT search CLI and example notebook
  • Runtime-only storage option
  • Naive community merge using time period

Full Release Notes

## What's Changed
- minor: Add Incremental Indexing
- minor: Added DRIFT graph reasoning query module
- minor: embeddings moved to a different workflow
- minor: Add DRIFT search cli and example notebook
- patch: Add config for incremental updates
- patch: Add embeddings to subflow.
- patch: Add naive community merge using time period
- patch: Add relationship merge
- patch: Add runtime-only storage option.
- patch: Add text units update
- patch: Allow empty workflow returns to avoid disk writing.
- patch: Apply pandas optimizations to create final entities
- patch: Calculate new inputs and deleted inputs on update
- patch: Collapse covariates flow.
- patch: Collapse create-base-entity-graph.
- patch: Collapse create-final-community-reports.
- patch: Collapse create-final-documents.
- patch: Collapse create-final-entities.
- patch: Collapse create-final-nodes.
- patch: Collapse create_base_documents.
- patch: Collapse create_base_text_units.
- patch: Collapse create_final_relationships.
- patch: Collapse entity extraction.
- patch: Collapse entity summarize.
- patch: Collapse intermediate workflow outputs.
- patch: Dependency updates
- patch: Extract DataShaper-less flows.
- patch: Fix Community ID loading for DRIFT search over existing indexes
- patch: Fix embeddings faulty assignments
- patch: Fix init defaults for vector store and drift img in docs
- patch: Fix nested json parsing
- patch: Fix some edge cases on Drift Search over small input sets
- patch: Fix var name for embedding
- patch: Merge existing and new entities, updating values accordingly
- patch: Merge text_embed into create-final-relationships subflow.
- patch: Move embedding verbs to operations.
- patch: Moving verbs around.
- patch: Optimize Create Base Documents subflow
- patch: Optimize text unit relationship count
- patch: Perf optimizations in map_query_to_entities()
- patch: Remove aggregate_df from final coomunities and final text units
- patch: Remove duplicated relationships and nodes
- patch: Remove unused column from final entities
- patch: Reorganized api,reporter,callback code into separate components. Defined debug profiles.
- patch: Small cleanup in community context history building
- patch: Transient entity graph and snapshotting.
- patch: Update Incremental Indexing to new embeddings workflow
- patch: Use mkdocs for documentation
- patch: add backwards compatibility patch to vector store.
- patch: add-autogenerated-cli-docs
- patch: fix docs image path
- patch: refactor use of vector stores and update support for managed identity
- patch: remove redundant error-handling code from global-search
- patch: reorganize cli layer

**Full Changelog**: https://github.com/microsoft/graphrag/compare/v0.3.6...v0.4.0