Starstruk Logo MarkStarstruk

v7.0.0

neuml/txtaiv7.0.0Feb 21, 2024by davidmezzetti

AI Summary

This major release introduces the next generation of semantic graph capabilities, including graph search, advanced traversal, RAG, and API security features.

Key Highlights

  • Add indexing of embeddings graph relationships
  • Expand graph capabilities for advanced graph traversal
  • Add feature to return embeddings search results as graph
  • Add support for binary content via API
  • Add documentation for API security

New Features

  • Semantic Graph indexing
  • Graph search results via API
  • Graph RAG
  • Binary content support
  • API security and authorization
  • LoRA/QLoRA training support

Full Release Notes

### 🎉 We're excited to announce the release of txtai 7.0 🎉

_If you like txtai, please remember to give it a ⭐!_ 

7.0 introduces the next generation of the semantic graph. This release adds support for graph search, advanced graph traversal and graph RAG. It also adds binary support to the API, index format improvements and training LoRA/QLoRA models. See below for more.

New Features
--------------------------
- Add indexing of embeddings graph relationships (#525)
- Expand the graph capabilities to enable advanced graph traversal (#534, #540)
- Add feature to return embeddings search results as graph (#644)
- Add RAG with Semantic Graphs notebook (#645)
- Graph search results via API (#670)
- Add knowledge graphs via LLM-driven entity extraction notebook (#671)
- Add advanced RAG with graph path traversal notebook (#672) 
- Add support for binary content via API (#630)
- Add MessagePack encoding to API (#658)
- Add documentation for API security (#627)
- Add notebook that covers API authorization and authentication (#628)
- Add top level import for LLM (#648)
- Add external vectorization notebook (#651)
- Add configuration override to embeddings.load (#657)
- Add what's new in txtai 7.0 notebook (#673)

Improvements
--------------------------
- Benchmark script improvements (#641)
- ImportError: Textractor pipeline is not available - install "pipeline" extra to enable (#646)
- Resolve external vector transform functions (#650)
- Change default embeddings config format to json (#652)
- Store index ids outside of configuration when content is disabled (#653)
- Update HFTrainer to add PEFT support (#654)
- Update 40_Text_to_Speech_Generation.ipynb (#666)- thank you @babinux
- Adding training dependencies to notebooks (#669)

Bug Fixes
--------------------------
- Fix various issues with subindex reloading (#618)
- Fix benchmarks script (#636)
- Set tokenizer.pad_token when empty for all training paths (#649)
- Fix documentation code filters (#656)
- Issues with NetworkX when using graph subindex (#664)

A big thank you goes to Jordan Matelsky (@j6k4m8) for his help in integrating the [GrandCypher](https://github.com/aplbrain/grand-cypher) library into txtai!