v0.13.0
infiniflow/ragflowv0.13.0Oct 31, 2024by KevinHuSh
AI Summary
Released on October 31, 2024, this update adds team support, HTTP APIs, and a Python SDK. It introduces smart chunking for Markdown, context retrieval via entity extraction, and the Invoke operator for Agents. It also integrates with Dify and ChatGPT-on-WeChat.
Key Highlights
- Added team support.
- Upgrade UI for Agent.
- Added HTTP API for knowledge base, document parsing, dialogs, etc., with a Python SDK.
- Added smart chunking for Markdown.
- Added context retrieval through entity extraction from LLM.
New Features
- Added Invoke operator for Agent.
- Added support for DIFY external knowledge base API.
- Added integration for ChatGPT-on-WeChat.
- Added support for GLM4-9B and Yi-Lightning models.
Full Release Notes
## Summary - Added team support - Upgrade UI for Agent - Added HTTP API for knowledge base, document parsing, dialogs, etc. Python SDK is also provided, which can be installed via `pip install ragflow-sdk==0.13.0`. - Added smart chunking for Markdown - Added context retrieval through entity extraction from LLM - Added Invoke operator for Agent - Added support for DIFY external knowledge base API - Added integration for ChatGPT-on-WeChat - Added support for GLM4-9B, Yi-Lightning ## What's Changed * added tests.yml by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2685 * fix tests.yml by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2688 * refine readme by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2689 * format mind_map_extractor code by @yqkcn in https://github.com/infiniflow/ragflow/pull/2686 * Simplify the usage of dict by @yqkcn in https://github.com/infiniflow/ragflow/pull/2681 * remove unused import by @yqkcn in https://github.com/infiniflow/ragflow/pull/2679 * Refactoring entity_resolution by @guoyuhao2330 in https://github.com/infiniflow/ragflow/pull/2692 * fix error in exception by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2694 * Updated Dockerfile by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2695 * fix: Fixed the issue where no error message was displayed when uploading a file that was too large #2258 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2697 * Updated Dockefile to use cache by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2703 * Fix Dockerfile for arm64 by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2705 * Updated Build Docker Image by @writinwaters in https://github.com/infiniflow/ragflow/pull/2706 * Fix Dockerfile.slim by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2708 * Fix poetry cache by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2709 * docs: Migrate README to Compose V2 syntax by @anud18 in https://github.com/infiniflow/ragflow/pull/2711 * build multi-arch image by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2710 * Fix/bedrock issues by @Omarlsg98 in https://github.com/infiniflow/ragflow/pull/2718 * remove redeclared get_json_result func by @yqkcn in https://github.com/infiniflow/ragflow/pull/2675 * fix re.escape problem by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2735 * chore: update operators.py by @eltociear in https://github.com/infiniflow/ragflow/pull/2724 * support sequence2txt and tts model in Xinference by @JobSmithManipulation in https://github.com/infiniflow/ragflow/pull/2696 * Add query parts to lamp by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2736 * feat: Add component TuShare #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2745 * Fix LIGHTEN. Close #2723 by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2744 * expand rerank range by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2746 * Modified download_deps.py by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2747 * Update azure_sas_conn.py - fixing container_url typo by @jstride in https://github.com/infiniflow/ragflow/pull/2740 * solve knowledgegraph issue when calling gemini model by @JobSmithManipulation in https://github.com/infiniflow/ragflow/pull/2738 * move import positions by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2753 * feat: The same query conditions on the search page should not request the interface every time the mind map drawer is opened. #2759 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2760 * Initial draft of Python APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2719 * Fix component exesql by @guoyuhao2330 in https://github.com/infiniflow/ragflow/pull/2754 * fix xinference add rerank model bug by @wangguo1230 in https://github.com/infiniflow/ragflow/pull/2758 * Fix docx images by @guoyuhao2330 in https://github.com/infiniflow/ragflow/pull/2756 * Docs:fix user guide links by @Howe829 in https://github.com/infiniflow/ragflow/pull/2761 * feat: Add a note node to the agent canvas #2767 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2768 * HTTP API documents, part1 by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2713 * Use slim image as the default by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2772 * Fix README description by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2773 * Fix README and some comments by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2774 * fix: Large document thumbnail display failed by @chongchuanbing in https://github.com/infiniflow/ragflow/pull/2763 * fix bedrock issue by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2776 * fix: torch dependency start error by @chongchuanbing in https://github.com/infiniflow/ragflow/pull/2777 * Fix error commands by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2778 * trival by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2779 * Refactor README on different docker version. by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2775 * refine agent by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2787 * Updated instructions on downloading RAGFlow Slim and RAGFlow all-in-one. by @writinwaters in https://github.com/infiniflow/ragflow/pull/2785 * fix: Change document status with @tanstack/react-query #13306 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2788 * Refactor Dataset API by @Feiue in https://github.com/infiniflow/ragflow/pull/2783 * feat: Arrange the form of generate, categorize, switch, retrieval operators vertically #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2800 * support api-version and change default-model in adding azure-openai and openai by @JobSmithManipulation in https://github.com/infiniflow/ragflow/pull/2799 * fix: document preview by @chongchuanbing in https://github.com/infiniflow/ragflow/pull/2795 * reduce rerank batch size by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2801 * Fixed broken links. by @writinwaters in https://github.com/infiniflow/ragflow/pull/2805 * support chat model in huggingface by @JobSmithManipulation in https://github.com/infiniflow/ragflow/pull/2802 * fix LIGHTEN issue by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2806 * trival by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2808 * fix chat and thumbnail bug by @chongchuanbing in https://github.com/infiniflow/ragflow/pull/2803 * Refactor Chat API by @Feiue in https://github.com/infiniflow/ragflow/pull/2804 * refine components retrieval and rewrite by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2818 * Refactor API for document and session by @Feiue in https://github.com/infiniflow/ragflow/pull/2819 * Updated dataset APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2820 * refine token similarity by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2824 * feat: Add message_history_window_size to CategorizeForm #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2826 * feat: Fix translation issue of message_history_window_size #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2828 * Refactor Document API by @Feiue in https://github.com/infiniflow/ragflow/pull/2833 * Updated chat APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2831 * let presentation do raptor by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2838 * feat: Generate operator names in an auto-incremental manner #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2844 * refine API token application by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2847 * refactor api util by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2849 * add rm token by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2850 * Feat:Compatible with Dify's External Knowledge API by @Howe829 in https://github.com/infiniflow/ragflow/pull/2848 * Enhance email validation: Allow top-level domains with 5 letters by @mrnicegyu11 in https://github.com/infiniflow/ragflow/pull/2856 * refactor by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2858 * Fix keys of Xinference deployed models, especially has the same model name with public hosted models. by @0000sir in https://github.com/infiniflow/ragflow/pull/2832 * feat: API access key management #2846 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2865 * add API for tenant function by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2866 * Refactor Chunk API by @Feiue in https://github.com/infiniflow/ragflow/pull/2855 * Updated session APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2868 * update dashscope by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2871 * feat: Configure the root directory alias #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2875 * feat: Load markdown file with "asset/source" #1739 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2880 * Fixed docker build by @yuzhichang in https://github.com/infiniflow/ragflow/pull/2881 * Draft: Updated file management-related APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2882 * Miscellaneous minor updates by @writinwaters in https://github.com/infiniflow/ragflow/pull/2885 * feat: Bind data to TenantTable #2846 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2883 * add team tag to kb by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2890 * Delete useless files by @Feiue in https://github.com/infiniflow/ragflow/pull/2889 * feat: Use Badge.Ribbon to distinguish the teams to which the knowledge base belongs #2846 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2891 * add owner check for team work by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2892 * fix: Fixed the issue of error reporting when uploading files in the chat box #2897 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2898 * DRAFT: Updated chunk APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2901 * DRAFT: Miscellaneous proofedits on Python APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/2903 * add nginx path for sdk handlers (#2899) by @kaleocheng in https://github.com/infiniflow/ragflow/pull/2900 * DRAFT: Miscellaneous updates to HTTP API. Tried to finish off Python API ref… by @writinwaters in https://github.com/infiniflow/ragflow/pull/2909 * Resolves #2905 openai compatible model provider add llama.cpp rerank support by @ziyu4huang in https://github.com/infiniflow/ragflow/pull/2906 * add zhipu glm-4-9b by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2912 * Add agent component for web crawler by @wingjson in https://github.com/infiniflow/ragflow/pull/2878 * Feature/feat1017 by @chongchuanbing in https://github.com/infiniflow/ragflow/pull/2872 * feat: Replace crawler icon #2915 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2916 * Fix some issues in API by @Feiue in https://github.com/infiniflow/ragflow/pull/2902 * fix thumbnail issue by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2917 * add package crawl4ai by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2918 * bigger resolution for OCR by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2919 * DRAFT: Miscellaneous updates to HTTP API reference by @writinwaters in https://github.com/infiniflow/ragflow/pull/2923 * Fix thumbnail_img NoneType error by @wingjson in https://github.com/infiniflow/ragflow/pull/2941 * fix sequence2txt error and usage total token issue by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2961 * [Bug]: When use OpenAI chat model , raise ERROR: 'CompletionUsage' object has no attribute 'get' #2948 by @yqwang96 in https://github.com/infiniflow/ragflow/pull/2949 * add auto keywords and auto-question by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2965 * add component invoke by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2967 * make titles in markdown not be splited with following content by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2971 * [Bug]: unnecessary auto-increment calculations in the tokens statistics of the chat model by @yqwang96 in https://github.com/infiniflow/ragflow/pull/2969 * DRAFT: Updated python and http api references by @writinwaters in https://github.com/infiniflow/ragflow/pull/2973 * fix SILICONFLOW rerank error by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2980 * fix generate string join issue by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2983 * minor by @writinwaters in https://github.com/infiniflow/ragflow/pull/2984 * Fix some issues in API by @Feiue in https://github.com/infiniflow/ragflow/pull/2982 * feat: Added auto_keywords and auto_questions fields to the parsing configuration page #2687 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2987 * allowing docker container to access service on host by @benquike in https://github.com/infiniflow/ragflow/pull/2895 * refactor auto keywords and auto question by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/2990 * Updated HTTP API Reference (document, chat assistant, session, chat) by @writinwaters in https://github.com/infiniflow/ragflow/pull/2994 * Test Cases by @Feiue in https://github.com/infiniflow/ragflow/pull/2993 * Remove defaults to 'None' by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/2996 * Integrating RAGFlow API as a Plugin into ChatGPT-on-WeChat by @timebankspace in https://github.com/infiniflow/ragflow/pull/2988 * feat: Limit the maximum value of auto keywords to 30 #2687 by @cike8899 in https://github.com/infiniflow/ragflow/pull/2991 * minor by @writinwaters in https://github.com/infiniflow/ragflow/pull/2998 * Miscellaneous updates to HTTP and PYthon APIs by @writinwaters in https://github.com/infiniflow/ragflow/pull/3000 * Miscellaneous updates to HTTP API Reference by @writinwaters in https://github.com/infiniflow/ragflow/pull/3005 * Fix some issues in API and test by @Feiue in https://github.com/infiniflow/ragflow/pull/3001 * Fix issues in API by @Feiue in https://github.com/infiniflow/ragflow/pull/3008 * make gemini robust by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3012 * Final touches to HTTP and Python API references by @writinwaters in https://github.com/infiniflow/ragflow/pull/3019 * Fix non-null violation when inviting people to team by @b246wang in https://github.com/infiniflow/ragflow/pull/3015 * Minor editorial updates to the HTTP API reference by @writinwaters in https://github.com/infiniflow/ragflow/pull/3027 * add model moonshot-v1-auto by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3051 * fix baidufanyi param error by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3053 * Bump werkzeug from 3.0.3 to 3.0.6 by @dependabot in https://github.com/infiniflow/ragflow/pull/3026 * Turn resource to plural form by @Feiue in https://github.com/infiniflow/ragflow/pull/3059 * feat: Add hint for operators, round to square, input variable, readable operator ID. #3056 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3057 * Turn resource to plural form by @Feiue in https://github.com/infiniflow/ragflow/pull/3061 * agent template upgrade by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3060 * remove file size check for sdk API by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3066 * feat: Add component Invoke #2908 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3067 * refactor error message of qwen by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3074 * fix local variable ans by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3077 * Update 'api key' by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/3078 * feat: If the model supplier is not set, click the OK button to jump directly to the page for setting the model supplier. #3068 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3069 * fix: modify the response of metadata in Dify retrieval api by @k0286 in https://github.com/infiniflow/ragflow/pull/3076 * search between multiple indiices for team function by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3079 * feat: Add InvokeNode #1908 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3081 * let 'Generate' take user's input as parameter by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3086 * feat: Allow the component id drop-down box to select the answer operator #3085 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3087 * feat: The order of the category operator form is messed up after refreshing the page #3088 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3089 * Updated HTTP API reference and Python API reference based on test results by @writinwaters in https://github.com/infiniflow/ragflow/pull/3090 * Update README by @JinHai-CN in https://github.com/infiniflow/ragflow/pull/3092 * Updated parser_config description by @writinwaters in https://github.com/infiniflow/ragflow/pull/3104 * Fix bugs in API by @Feiue in https://github.com/infiniflow/ragflow/pull/3103 * Differentiated API key names to avoid confusion. by @writinwaters in https://github.com/infiniflow/ragflow/pull/3107 * Added chunk methods by @writinwaters in https://github.com/infiniflow/ragflow/pull/3110 * docs updates for 0.13 release by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3108 * Publish RAGFlow's HTTP and Python API references by @writinwaters in https://github.com/infiniflow/ragflow/pull/3116 * Tried to fix a link issue by @writinwaters in https://github.com/infiniflow/ragflow/pull/3117 * Manage ragflow-sdk with poetry by @yuzhichang in https://github.com/infiniflow/ragflow/pull/3115 * add yi-lightning by @KevinHuSh in https://github.com/infiniflow/ragflow/pull/3119 * Fixed a docusaurus display issue by @writinwaters in https://github.com/infiniflow/ragflow/pull/3120 * feat: Delete http_api_reference.md from api folder #1102 by @cike8899 in https://github.com/infiniflow/ragflow/pull/3121 * Fixed a docusaurus display issue. by @writinwaters in https://github.com/infiniflow/ragflow/pull/3124 * fix: remove useless test code by @zhyg in https://github.com/infiniflow/ragflow/pull/3122 * Fixed a Docusaurus display issue. by @writinwaters in https://github.com/infiniflow/ragflow/pull/3125 * Add test for CI by @Feiue in https://github.com/infiniflow/ragflow/pull/3114 * Updated list_documents() by @writinwaters in https://github.com/infiniflow/ragflow/pull/3126 * Moved the Upgrade Manuel out of FAQ by @writinwaters in https://github.com/infiniflow/ragflow/pull/3131 * Reworded descriptions for development versions and latest version by @writinwaters in https://github.com/infiniflow/ragflow/pull/3132 * minor by @writinwaters in https://github.com/infiniflow/ragflow/pull/3137 ## New Contributors * @anud18 made their first contribution in https://github.com/infiniflow/ragflow/pull/2711 * @Omarlsg98 made their first contribution in https://github.com/infiniflow/ragflow/pull/2718 * @jstride made their first contribution in https://github.com/infiniflow/ragflow/pull/2740 * @wangguo1230 made their first contribution in https://github.com/infiniflow/ragflow/pull/2758 * @Howe829 made their first contribution in https://github.com/infiniflow/ragflow/pull/2761 * @chongchuanbing made their first contribution in https://github.com/infiniflow/ragflow/pull/2763 * @mrnicegyu11 made their first contribution in https://github.com/infiniflow/ragflow/pull/2856 * @0000sir made their first contribution in https://github.com/infiniflow/ragflow/pull/2832 * @kaleocheng made their first contribution in https://github.com/infiniflow/ragflow/pull/2900 * @ziyu4huang made their first contribution in https://github.com/infiniflow/ragflow/pull/2906 * @yqwang96 made their first contribution in https://github.com/infiniflow/ragflow/pull/2949 * @benquike made their first contribution in https://github.com/infiniflow/ragflow/pull/2895 * @timebankspace made their first contribution in https://github.com/infiniflow/ragflow/pull/2988 * @b246wang made their first contribution in https://github.com/infiniflow/ragflow/pull/3015 * @k0286 made their first contribution in https://github.com/infiniflow/ragflow/pull/3076 * @zhyg made their first contribution in https://github.com/infiniflow/ragflow/pull/3122 **Full Changelog**: https://github.com/infiniflow/ragflow/compare/v0.12.0...v0.13.0