v2.4.14
milvus-io/milvusv2.4.14Oct 29, 2024by yanliang567
AI Summary
Milvus 2.4.14 addresses a critical issue from version 2.4.13 that could cause collection information loss after garbage collection and fixes resource leaks, while optimizing delete and compaction performance.
Key Highlights
- Fixed collection info recovery after snapshotKV garbage collection.
- Enabled parallel execution of L0 compactions.
- Optimized delete and compaction performance for large-scale operations.
- Support for memory mode chunk cache.
New Features
- Memory mode chunk cache support
- Support for DB in bulkinsert
Full Release Notes
## v2.4.14 Release Date: October 31, 2024 | Milvus version | Python SDK version | Java SDK version | Node.js SDK version | |----------------|--------------------|------------------|---------------------| | 2.4.14 | 2.4.9 | 2.4.7 | 2.4.9 | Milvus 2.4.14 addresses a critical issue from version 2.4.13 that could cause collection information to be lost after `snapshotKV` garbage collection. It also fixed a couple of resource leaks. Additionally, this release includes numerous enhancements focused on improving stability in large-scale delete operations and compaction performance. ### Features - Supported memory mode chunk cache ([#35836](https://github.com/milvus-io/milvus/pull/35836)) - Supported db for bulkinsert ([#37017](https://github.com/milvus-io/milvus/pull/37017)) ### Improvements - Delete/Compaction Optimization - Enabled parallel execution of l0 compactions ([#36985](https://github.com/milvus-io/milvus/pull/36985)) - Batched forward delete when using direct forward ([#37107](https://github.com/milvus-io/milvus/pull/37107)) - Skipped loading delta data in delegater when using remoteload ([#37112](https://github.com/milvus-io/milvus/pull/37112)) - Directly forwarded delta excluding l0 segments ([#36914](https://github.com/milvus-io/milvus/pull/36914)) - Added prioritization of compaction tasks in DataCoord ([#36979](https://github.com/milvus-io/milvus/pull/36979)) - Tracked complex delete rates ([#36958](https://github.com/milvus-io/milvus/pull/36958)) - Refactored CreateCollection in RESTFul API ([#36885](https://github.com/milvus-io/milvus/pull/36885)) - Fused multiple 'and' and 'or' operations into a single op ([#36973](https://github.com/milvus-io/milvus/pull/36973)) - Made skip load work for all branches ([#37161](https://github.com/milvus-io/milvus/pull/37161)) - Upgraded Minio dependency to support EKS Pod Identities ([#37089](https://github.com/milvus-io/milvus/pull/37089)) - Tidied import options ([#37078](https://github.com/milvus-io/milvus/pull/37078)) - Limited maximum number of import jobs ([#36892](https://github.com/milvus-io/milvus/pull/36892)) - Preallocated data slice to avoid re-allocating memory ([#37044](https://github.com/milvus-io/milvus/pull/37044)) - Prevented DataNode from loading the bf ([#37027](https://github.com/milvus-io/milvus/pull/37027)) - Avoided limiting ddl operations repeatedly ([#37011](https://github.com/milvus-io/milvus/pull/37011)) - Made the configuration item `datanode.import.maxconcurrenttasknum` dynamically adjustable ([#37103](https://github.com/milvus-io/milvus/pull/37103)) - Used `queryNode.mmap.growingMmapEnabled` to control the behavior of interim index ([#36391](https://github.com/milvus-io/milvus/pull/36391)) - Populated the `Level` and `StartPosition` fields in segmentLoadInfo of growing segment ([#36911](https://github.com/milvus-io/milvus/pull/36911)) - Forced to stop buffer messages when receiving the drop collection message ([#36917](https://github.com/milvus-io/milvus/pull/36917)) - Added metrics for querynode delete buffer info ([#37097](https://github.com/milvus-io/milvus/pull/37097)) - Added collection name label for some metric ([#37159](https://github.com/milvus-io/milvus/pull/37159)) - Used middleware to observe RESTful v2 in/out rpc stats ([#37224](https://github.com/milvus-io/milvus/pull/37224)) - Changed GPU default memory pool size ([#36969](https://github.com/milvus-io/milvus/pull/36969)) - Updated Knowhere version to 2.3.12 ([#37132](https://github.com/milvus-io/milvus/pull/37132)) - Allowed deleting data when disk quota exhausted ([#37139](https://github.com/milvus-io/milvus/pull/37139)) ### Bug fixes - Fixed collection info that could not be recovered from metakv after restart if all snapshots were garbage collected ([#36950](https://github.com/milvus-io/milvus/pull/36950)) - Corrected the rpc error code to avoid invalid retry in client ([#37025](https://github.com/milvus-io/milvus/pull/37025)) - Ignored db not found error in quota center ([#36850](https://github.com/milvus-io/milvus/pull/36850)) - Fixed goroutine leakage in QueryNode by using singleton delete pool ([#37225](https://github.com/milvus-io/milvus/pull/37225)) - Fixed collection leak in querynode ([#37079](https://github.com/milvus-io/milvus/pull/37079)) - Fixed leakage of clustering compaction task ([#36803](https://github.com/milvus-io/milvus/pull/36803)) - Prohibited renaming a collection that had an alias ([#37208](https://github.com/milvus-io/milvus/pull/37208)) - Made sure alias was cached ([#36808](https://github.com/milvus-io/milvus/pull/36808)) - Search/query could have failed during updating delegator cache ([#37174](https://github.com/milvus-io/milvus/pull/37174)) - Excluded l0 compaction when clustering was executing ([#37142](https://github.com/milvus-io/milvus/pull/37142)) - Referenced collection meta when loading l0 segment meta only ([#37179](https://github.com/milvus-io/milvus/pull/37179)) - Delegator might have become unserviceable after querycoord restart ([#37100](https://github.com/milvus-io/milvus/pull/37100)) - Dynamic release partition might have failed search/query ([#37099](https://github.com/milvus-io/milvus/pull/37099)) - Rectified delete buffer row count quota value ([#37068](https://github.com/milvus-io/milvus/pull/37068)) - Passed full field list when partial load enabled ([#37063](https://github.com/milvus-io/milvus/pull/37063)) - Query node panic occurred during sending rpc to worker ([#36988](https://github.com/milvus-io/milvus/pull/36988)) - Datacoord got stuck at stopping progress ([#36961](https://github.com/milvus-io/milvus/pull/36961)) - Fixed the out-of-bounds access in the growing segment when raw data was replaced by interim index ([#36938](https://github.com/milvus-io/milvus/pull/36938)) - Rootcoord got stuck at graceful stop progress ([#36881](https://github.com/milvus-io/milvus/pull/36881))