0.4.0
pingcap/autoflow0.4.0Jan 3, 2025by Mini256
AI Summary
This release introduces the Evaluation (beta) tool for measuring Chat Engine output quality with Factual Correctness and Semantic Similarity metrics, along with support for new LLM and embedding model providers including Gitee AI, vLLM, Xinference, and Amazon Bedrock.
Key Highlights
- Added Evaluation (beta) tool with Factual Correctness and Semantic Similarity metrics
- Support for new LLM providers: Gitee AI, vLLM, and Xinference
- Support for new embedding model providers: Gitee AI and Amazon Bedrock
- Added file upload size limit (10MB default via max_upload_file_size)
- Added ability to download reference files in chat page
New Features
- Evaluation tool for measuring Chat Engine output performance
- Factual Correctness metric
- Semantic Similarity metric
- Gitee AI LLM provider support
- vLLM LLM provider support
- Xinference LLM provider support
- Gitee AI embedding model provider support
- Amazon Bedrock embedding model provider support
- File upload size limit configuration
- Reference file download in chat
Full Release Notes
## Highlights
- Support [Evaluation (beta)](https://tidb.ai/docs/evaluation) tool to evaluate the performance and reliability of the Chat Engine’s outputs (by @Icemap @634750802 ).
- Current support key metrics:
- Factual Correctness
- Semantic Similarity
- Support new LLM providers
- [Gitee AI](https://tidb.ai/docs/llm#gitee-ai) (by @lc-soft)
- Test new OpenAI-like providers (by @jrj5423)
- [vLLM](https://tidb.ai/docs/llm#vllm)
- [Xinference](https://tidb.ai/docs/llm#xinference)
- Support new embedding model providers
- [Gitee AI](https://tidb.ai/docs/embedding-model#gitee-ai) (by @lc-soft)
- [Amazon Bedrock](https://tidb.ai/docs/embedding-model#amazon-bedrock) (by @jrj5423 @Mini256)
## Improvements
- Limit the upload file size via `max_upload_file_size` (10MB by default) parameter on site setting (by @sszgwdk @634750802 )
- Support download the reference file in the chat page (by @Rutheniumlmw @634750802 )
## New Contributors
* @sszgwdk made their first contribution in https://github.com/pingcap/autoflow/pull/497
* @jrj5423 made their first contribution in https://github.com/pingcap/autoflow/pull/522
* @Rutheniumlmw made their first contribution in https://github.com/pingcap/autoflow/pull/538
* @lc-soft made their first contribution in https://github.com/pingcap/autoflow/pull/470
* @nicognaW made their first contribution in https://github.com/pingcap/autoflow/pull/496
If you are deploying Autoflow using docker, please follow the [Upgrade](https://tidb.ai/docs/deploy-with-docker#upgrade) guide to upgrade your Autoflow.
**Full Changelog**: https://github.com/pingcap/autoflow/compare/v0.3.0...0.4.0