v0.5.1
NirDiamant/GenAI_Agentsv0.5.1Nov 21, 2024by northmachine
AI Summary
This patch release addresses user feedback by fixing Neo4j memory overflow issues and adding support for Word documents. It also introduces a project deletion API and model concurrency controls.
Key Highlights
- Support for Word Documents (.doc/.docx)
- New Project Deletion API
- Model call concurrency setting (default 5)
- Improved logging with startup success markers
- Fixed Neo4j memory overflow issues
New Features
- Word document support for knowledge base construction
- Project deletion API to clear projects and data
- Model concurrency control for large-scale construction
- Startup success marker in logs
- Fixed concurrent Neo4j query execution and GDS library conflicts
- Auto-fill default Neo4j password for project creation/ modification
- Frontend bug fixes (embedded JS dependencies)
Full Release Notes
# Version 0.5.1 (2024-11-21) OpenSPG released version v0.5.1 on November 21, 2024. This version focuses on addressing user feedback and introduces a series of new features and user experience optimizations. --- ### 🌟 **New Features** 1. **Support for Word Documents** - Users can now directly upload `.doc` or `.docx` files to streamline the knowledge base construction process. <img src="https://github.com/user-attachments/assets/86ad11d8-42ed-44f4-91ab-f9a7c6346df2" width="600" > 2. **New Project Deletion API** - Quickly clear and delete projects and related data through an API, compatible with the latest Neo4j image version. 3. **Model Call Concurrency Setting** - Added the `builder.model.execute.num` parameter, with a default concurrency of 5, to improve efficiency in large-scale knowledge base construction. <img src="https://github.com/user-attachments/assets/ac7653bd-bf0c-464f-839b-8385ae6fb2c2" width="600" > 4. **Improved Logging** - Added a startup success marker in the logs to help users quickly verify if the service is running correctly. <img src="https://github.com/user-attachments/assets/56d42e84-d6c7-4743-a50c-5bf38fc87f58" width="600" > --- ### ⚙️ **User Experience Optimizations** 1. **Neo4j Memory Overflow Issues** - Addressed memory overflow problems in Neo4j during large-scale data processing, ensuring stable operation for extensive datasets. 2. **Concurrent Neo4j Query Execution Issues** - Optimized execution strategies to resolve Graph Data Science (GDS) library conflicts or failures in high-concurrency scenarios. 3. **Schema Preview Prefix Issue** - Fixed issues where extracted schema preview entities lacked necessary prefixes, ensuring consistency between extracted entities and predefined schemas. 4. **Default Neo4j Password for Project Creation/Modification** - Automatically fills a secure default password if none is specified during project creation or modification, simplifying the configuration process. 5. **Frontend Bug Fixes** - Resolved issues with JS dependencies relying on external addresses and embedded all frontend files into the image. Improved the knowledge base management interface for a smoother user experience. 6. **Empty Node/Edge Type in Neo4j Writes** - Enhanced writing logic to handle empty node or edge types during knowledge graph construction, preventing errors or data loss in such scenarios. # Version 0.5.1 (2024-11-21) OpenSPG 在 2024 年 11 月 21 日发布了 v0.5.1 版本。此版本重点解决了用户反馈的问题,并带来了一系列新功能和用户体验的优化。 ### 🌟 **新增功能** 1. **支持 word 文档的构建** - 用户现可通过知识库管理页面直接上传 .doc 或 .docx 后缀的文件,进行知识库的构建流程。这一更新使得知识内容的导入更加便捷,提高效率。 <img src="https://github.com/user-attachments/assets/9efec8d6-4b22-4680-86bb-9bf28d55d17b" width="600" > 2. **提供项目删除接口** - 为了帮助用户更高效地管理项目,我们新增了一个项目删除接口。用户可以通过访问 http://127.0.0.1:8887/project/api/delete?projectId=xx 完成项目的快速清空与删除操作。该接口会同步清理项目下的所有schema、知识库任务、知识库问答任务以及关联的 Neo4j 数据库。 Tips:使用此功能前,需确保已将 openspg-neo4j 镜像更新至最新版本 3. **支持模型调用并发度设置** - 在大规模知识库构建过程中,为了提高构建效率,我们引入了模型调用的并发控制机制。用户可以通过设置 builder.model.execute.num 参数来调整并发数量,默认值设定为5。这有助于避免因模型服务性能瓶颈而导致的任务失败或系统卡顿。 <img src="https://github.com/user-attachments/assets/ac7653bd-bf0c-464f-839b-8385ae6fb2c2" width="600" > 4. **日志中添加启动成功标识** - 为了让用户能够更直观地判断 OpenSPG 服务是否启动成功,我们在日志输出中加入了明确的启动成功标识。openspg-server 成功启动后,会输出这一标识。 <img src="https://github.com/user-attachments/assets/56d42e84-d6c7-4743-a50c-5bf38fc87f58" width="600" > ### ⚙️ **用户体验优化** 1. **解决大规模数据构建下 Neo4j 调用内存超限问题** - 针对在处理大规模数据集时出现的 Neo4j 内存溢出问题,我们进行了深入分析并实施了有效的解决方案。现在,面对大规模数据集Neo4j 能保持稳定运行,有效防止了因内存不足而导致的服务中断。 2. **解决多并发下执行 Neo4j 查询导致的 GDS 加载问题** - 在多并发场景下执行 Neo4j 查询时,图数据科学 (GDS) 库的加载会出现冲突或失败的情况。为此,我们优化了查询执行策略,确保了在高并发环境下的查询性能和稳定性。 3. **解决抽取结果 Schema 预览实体无前缀问题** - 在之前版本中,部分用户反馈在查看抽取结果的 Schema 预览时,实体名称缺少必要的前缀信息导致抽取的实体和预定义的Schema不一致。此次更新修正了这一问题,保证了所有实体名称的完整性和准确性。 4. **创建修改项目时 Neo4j 无密码时填充默认值** - 当用户在创建或修改项目时,如果未指定 Neo4j 密码,系统将自动填充一个安全的默认值,从而简化了配置流程,减少了用户的输入负担。 5. **前端 bugfix** - 修复了JS依赖外部地址问题,已将前端文件全部内置到镜像内;同时针对知识库管理页面进行了多项改进,以提供更加流畅的操作体验。 6. **解决点边类型为空导致的 Neo4j 写入失败问题** - 对于在构建知识图谱时可能出现的节点或关系类型为空的情况,我们优化了写入逻辑,确保即便在这些特殊情况下也能顺利完成数据的写入操作,避免了因类型缺失而引发的数据丢失或错误。