v0.13.0

infiniflow/ragflowv0.13.0Oct 31, 2024by KevinHuSh

AI Summary

Released on October 31, 2024, this update adds team support, HTTP APIs, and a Python SDK. It introduces smart chunking for Markdown, context retrieval via entity extraction, and the Invoke operator for Agents. It also integrates with Dify and ChatGPT-on-WeChat.

Key Highlights

  • Added team support.
  • Upgrade UI for Agent.
  • Added HTTP API for knowledge base, document parsing, dialogs, etc., with a Python SDK.
  • Added smart chunking for Markdown.
  • Added context retrieval through entity extraction from LLM.

New Features

  • Added Invoke operator for Agent.
  • Added support for DIFY external knowledge base API.
  • Added integration for ChatGPT-on-WeChat.
  • Added support for GLM4-9B and Yi-Lightning models.

Full Release Notes

## Summary
- Added team support
- Upgrade UI for Agent
- Added HTTP API for knowledge base, document parsing, dialogs, etc. Python SDK is also provided, which can be installed via `pip install ragflow-sdk==0.13.0`.
- Added smart chunking for Markdown
- Added context retrieval through entity extraction from LLM
- Added Invoke operator for Agent
- Added support for DIFY external knowledge base API
- Added integration for ChatGPT-on-WeChat
- Added support for GLM4-9B, Yi-Lightning


## What's Changed
* added tests.yml by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2685
* fix tests.yml by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2688
* refine readme by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2689
* format mind_map_extractor code by @yqkcn in https://github.com/infiniflow/ragflow/pull/2686
* Simplify the usage of dict by @yqkcn in https://github.com/infiniflow/ragflow/pull/2681
* remove unused import by @yqkcn in https://github.com/infiniflow/ragflow/pull/2679
* Refactoring entity_resolution by @guoyuhao2330 in https://github.com/infiniflow/ragflow/pull/2692
* fix error in exception by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2694
* Updated Dockerfile by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2695
* fix: Fixed the issue where no error message was displayed when uploading a file that was too large #2258 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2697
* Updated Dockefile to use cache by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2703
* Fix Dockerfile for arm64 by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2705
* Updated Build Docker Image by @writinwaters in https://github.com/infiniflow/ragflow/pull/2706
* Fix Dockerfile.slim by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2708
* Fix poetry cache by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2709
* docs: Migrate README to Compose V2 syntax by @anud18 in https://github.com/infiniflow/ragflow/pull/2711
* build multi-arch image by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2710
* Fix/bedrock issues by @Omarlsg98 in https://github.com/infiniflow/ragflow/pull/2718
* remove redeclared get_json_result func by @yqkcn in https://github.com/infiniflow/ragflow/pull/2675
* fix re.escape problem by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2735
* chore: update operators.py by @eltociear in https://github.com/infiniflow/ragflow/pull/2724
* support sequence2txt and tts model in Xinference by @JobSmithManipulation in https://github.com/infiniflow/ragflow/pull/2696
* Add query parts to lamp by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2736
* feat: Add component  TuShare  #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2745
* Fix LIGHTEN. Close #2723 by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2744
* expand rerank range by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2746
* Modified download_deps.py by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2747
* Update azure_sas_conn.py - fixing container_url typo by @jstride in https://github.com/infiniflow/ragflow/pull/2740
* solve knowledgegraph issue when calling gemini model by @JobSmithManipulation in https://github.com/infiniflow/ragflow/pull/2738
* move import positions by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2753
* feat: The same query conditions on the search page should not request the interface every time the mind map drawer is opened. #2759 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2760
* Initial draft of Python APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2719
* Fix component exesql by @guoyuhao2330 in https://github.com/infiniflow/ragflow/pull/2754
* fix xinference add rerank model bug by @wangguo1230 in https://github.com/infiniflow/ragflow/pull/2758
* Fix docx images by @guoyuhao2330 in https://github.com/infiniflow/ragflow/pull/2756
* Docs:fix user guide links by @Howe829 in https://github.com/infiniflow/ragflow/pull/2761
* feat: Add a note node to the agent canvas #2767 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2768
* HTTP API documents, part1 by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2713
* Use slim image as the default by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2772
* Fix README description by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2773
* Fix README and some comments by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2774
* fix: Large document thumbnail display failed by @chongchuanbing in https://github.com/infiniflow/ragflow/pull/2763
* fix bedrock issue by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2776
* fix: torch dependency start error by @chongchuanbing in https://github.com/infiniflow/ragflow/pull/2777
* Fix error commands by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2778
* trival by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2779
* Refactor README on different docker version. by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2775
* refine agent by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2787
* Updated instructions on downloading RAGFlow Slim and RAGFlow all-in-one. by @writinwaters in https://github.com/infiniflow/ragflow/pull/2785
* fix: Change document status with @tanstack/react-query #13306 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2788
* Refactor Dataset API by @Feiue in https://github.com/infiniflow/ragflow/pull/2783
* feat: Arrange the form of generate, categorize, switch, retrieval operators vertically #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2800
* support api-version and change default-model in adding azure-openai and openai by @JobSmithManipulation in https://github.com/infiniflow/ragflow/pull/2799
* fix: document preview by @chongchuanbing in https://github.com/infiniflow/ragflow/pull/2795
* reduce rerank batch size by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2801
* Fixed broken links. by @writinwaters in https://github.com/infiniflow/ragflow/pull/2805
* support chat model in huggingface by @JobSmithManipulation in https://github.com/infiniflow/ragflow/pull/2802
* fix LIGHTEN issue by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2806
* trival by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2808
* fix chat and thumbnail bug by @chongchuanbing in https://github.com/infiniflow/ragflow/pull/2803
* Refactor Chat API by @Feiue in https://github.com/infiniflow/ragflow/pull/2804
* refine components retrieval and rewrite by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2818
* Refactor API for document and session by @Feiue in https://github.com/infiniflow/ragflow/pull/2819
* Updated dataset APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2820
* refine token similarity by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2824
* feat: Add message_history_window_size to CategorizeForm #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2826
* feat: Fix translation issue of message_history_window_size #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2828
* Refactor Document API by @Feiue in https://github.com/infiniflow/ragflow/pull/2833
* Updated chat APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2831
* let presentation do raptor by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2838
* feat: Generate operator names in an auto-incremental manner #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2844
* refine API token application by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2847
* refactor api util by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2849
* add rm token by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2850
* Feat:Compatible with Dify's External Knowledge API by @Howe829 in https://github.com/infiniflow/ragflow/pull/2848
* Enhance email validation: Allow top-level domains with 5 letters by @mrnicegyu11 in https://github.com/infiniflow/ragflow/pull/2856
* refactor by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2858
* Fix keys of Xinference deployed models, especially has the same model name with public hosted models. by @0000sir in https://github.com/infiniflow/ragflow/pull/2832
* feat: API access key management #2846 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2865
* add API for tenant function by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2866
* Refactor Chunk API by @Feiue in https://github.com/infiniflow/ragflow/pull/2855
* Updated session APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2868
* update dashscope by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2871
* feat: Configure the root directory alias #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2875
* feat: Load markdown file with "asset/source" #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2880
* Fixed docker build by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2881
* Draft: Updated file management-related APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2882
* Miscellaneous minor updates by @writinwaters in https://github.com/infiniflow/ragflow/pull/2885
* feat: Bind data to TenantTable #2846 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2883
* add team tag to kb by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2890
* Delete useless files by @Feiue in https://github.com/infiniflow/ragflow/pull/2889
* feat: Use Badge.Ribbon to distinguish the teams to which the knowledge base belongs #2846 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2891
* add owner check for team work by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2892
* fix: Fixed the issue of error reporting when uploading files in the chat box #2897 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2898
* DRAFT: Updated chunk APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2901
* DRAFT: Miscellaneous proofedits on Python APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2903
* add nginx path for sdk handlers (#2899) by @kaleocheng in https://github.com/infiniflow/ragflow/pull/2900
* DRAFT: Miscellaneous updates to HTTP API. Tried to finish off Python API ref… by @writinwaters in https://github.com/infiniflow/ragflow/pull/2909
* Resolves #2905 openai compatible model provider add llama.cpp rerank support by @ziyu4huang in https://github.com/infiniflow/ragflow/pull/2906
* add zhipu glm-4-9b by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2912
* Add agent component for  web crawler by @wingjson in https://github.com/infiniflow/ragflow/pull/2878
* Feature/feat1017 by @chongchuanbing in https://github.com/infiniflow/ragflow/pull/2872
* feat: Replace crawler icon #2915 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2916
* Fix some issues in API by @Feiue in https://github.com/infiniflow/ragflow/pull/2902
* fix thumbnail issue by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2917
* add package crawl4ai by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2918
* bigger resolution for OCR by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2919
* DRAFT: Miscellaneous updates to HTTP API reference by @writinwaters in https://github.com/infiniflow/ragflow/pull/2923
* Fix thumbnail_img NoneType error by @wingjson in https://github.com/infiniflow/ragflow/pull/2941
* fix sequence2txt error and usage total token issue by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2961
* [Bug]: When use OpenAI chat model , raise ERROR: 'CompletionUsage' object has no attribute 'get' #2948 by @yqwang96 in https://github.com/infiniflow/ragflow/pull/2949
* add auto keywords and auto-question by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2965
* add component invoke by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2967
* make titles in markdown not be splited with following content by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2971
* [Bug]: unnecessary auto-increment calculations in the tokens statistics of the chat model by @yqwang96 in https://github.com/infiniflow/ragflow/pull/2969
* DRAFT: Updated python and http api references by @writinwaters in https://github.com/infiniflow/ragflow/pull/2973
* fix SILICONFLOW rerank error by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2980
* fix generate string join issue by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2983
* minor by @writinwaters in https://github.com/infiniflow/ragflow/pull/2984
* Fix some issues in API by @Feiue in https://github.com/infiniflow/ragflow/pull/2982
* feat: Added auto_keywords and auto_questions fields to the parsing configuration page #2687 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2987
* allowing docker container to access service on host by @benquike in https://github.com/infiniflow/ragflow/pull/2895
* refactor auto keywords and auto question by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2990
* Updated HTTP API Reference (document, chat assistant, session, chat) by @writinwaters in https://github.com/infiniflow/ragflow/pull/2994
* Test Cases by @Feiue in https://github.com/infiniflow/ragflow/pull/2993
* Remove defaults to 'None' by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2996
* Integrating RAGFlow API as a Plugin into ChatGPT-on-WeChat by @timebankspace in https://github.com/infiniflow/ragflow/pull/2988
* feat: Limit the maximum value of auto keywords to 30 #2687 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2991
* minor by @writinwaters in https://github.com/infiniflow/ragflow/pull/2998
* Miscellaneous updates to HTTP and PYthon APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/3000
* Miscellaneous updates to HTTP API Reference by @writinwaters in https://github.com/infiniflow/ragflow/pull/3005
* Fix some issues in API and test by @Feiue in https://github.com/infiniflow/ragflow/pull/3001
* Fix issues in API by @Feiue in https://github.com/infiniflow/ragflow/pull/3008
* make gemini robust by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3012
* Final touches to HTTP and Python API references by @writinwaters in https://github.com/infiniflow/ragflow/pull/3019
* Fix non-null violation when inviting people to team by @b246wang in https://github.com/infiniflow/ragflow/pull/3015
* Minor editorial updates to the HTTP API reference by @writinwaters in https://github.com/infiniflow/ragflow/pull/3027
* add model moonshot-v1-auto by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3051
* fix baidufanyi param error by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3053
* Bump werkzeug from 3.0.3 to 3.0.6 by @dependabot in https://github.com/infiniflow/ragflow/pull/3026
* Turn resource to plural form by @Feiue in https://github.com/infiniflow/ragflow/pull/3059
* feat: Add hint for operators, round to square, input variable, readable operator ID.  #3056 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3057
* Turn resource to plural form by @Feiue in https://github.com/infiniflow/ragflow/pull/3061
* agent template upgrade by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3060
* remove file size check for sdk API by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3066
* feat: Add component Invoke #2908 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3067
* refactor error message of qwen by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3074
* fix local variable ans by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3077
* Update 'api key' by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/3078
* feat: If the model supplier is not set, click the OK button to jump directly to the page for setting the model supplier. #3068 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3069
* fix: modify the response of metadata in Dify retrieval api by @k0286 in https://github.com/infiniflow/ragflow/pull/3076
* search between multiple indiices for team function by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3079
* feat: Add InvokeNode #1908 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3081
* let 'Generate' take user's input as parameter by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3086
* feat: Allow the component id drop-down box to select the answer operator #3085 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3087
* feat: The order of the category operator form is messed up after refreshing the page #3088 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3089
* Updated HTTP API reference and Python API reference based on test results by @writinwaters in https://github.com/infiniflow/ragflow/pull/3090
* Update README by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/3092
* Updated parser_config description by @writinwaters in https://github.com/infiniflow/ragflow/pull/3104
* Fix bugs in API by @Feiue in https://github.com/infiniflow/ragflow/pull/3103
* Differentiated API key names to avoid confusion. by @writinwaters in https://github.com/infiniflow/ragflow/pull/3107
* Added chunk methods by @writinwaters in https://github.com/infiniflow/ragflow/pull/3110
* docs updates for 0.13 release by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3108
* Publish RAGFlow's HTTP and Python API references by @writinwaters in https://github.com/infiniflow/ragflow/pull/3116
* Tried to fix a link issue by @writinwaters in https://github.com/infiniflow/ragflow/pull/3117
* Manage ragflow-sdk with poetry by @yuzhichang in https://github.com/infiniflow/ragflow/pull/3115
* add yi-lightning by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3119
* Fixed a docusaurus display issue by @writinwaters in https://github.com/infiniflow/ragflow/pull/3120
* feat: Delete http_api_reference.md from  api folder #1102 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3121
* Fixed a docusaurus display issue. by @writinwaters in https://github.com/infiniflow/ragflow/pull/3124
* fix: remove useless test code by @zhyg in https://github.com/infiniflow/ragflow/pull/3122
* Fixed a Docusaurus display issue. by @writinwaters in https://github.com/infiniflow/ragflow/pull/3125
* Add test for CI by @Feiue in https://github.com/infiniflow/ragflow/pull/3114
* Updated list_documents() by @writinwaters in https://github.com/infiniflow/ragflow/pull/3126
* Moved the Upgrade Manuel out of FAQ by @writinwaters in https://github.com/infiniflow/ragflow/pull/3131
* Reworded descriptions for development versions and latest version by @writinwaters in https://github.com/infiniflow/ragflow/pull/3132
* minor by @writinwaters in https://github.com/infiniflow/ragflow/pull/3137

## New Contributors
* @anud18 made their first contribution in https://github.com/infiniflow/ragflow/pull/2711
* @Omarlsg98 made their first contribution in https://github.com/infiniflow/ragflow/pull/2718
* @jstride made their first contribution in https://github.com/infiniflow/ragflow/pull/2740
* @wangguo1230 made their first contribution in https://github.com/infiniflow/ragflow/pull/2758
* @Howe829 made their first contribution in https://github.com/infiniflow/ragflow/pull/2761
* @chongchuanbing made their first contribution in https://github.com/infiniflow/ragflow/pull/2763
* @mrnicegyu11 made their first contribution in https://github.com/infiniflow/ragflow/pull/2856
* @0000sir made their first contribution in https://github.com/infiniflow/ragflow/pull/2832
* @kaleocheng made their first contribution in https://github.com/infiniflow/ragflow/pull/2900
* @ziyu4huang made their first contribution in https://github.com/infiniflow/ragflow/pull/2906
* @yqwang96 made their first contribution in https://github.com/infiniflow/ragflow/pull/2949
* @benquike made their first contribution in https://github.com/infiniflow/ragflow/pull/2895
* @timebankspace made their first contribution in https://github.com/infiniflow/ragflow/pull/2988
* @b246wang made their first contribution in https://github.com/infiniflow/ragflow/pull/3015
* @k0286 made their first contribution in https://github.com/infiniflow/ragflow/pull/3076
* @zhyg made their first contribution in https://github.com/infiniflow/ragflow/pull/3122

**Full Changelog**: https://github.com/infiniflow/ragflow/compare/v0.12.0...v0.13.0