v0.4.7
microsoft/markitdownv0.4.7Jun 13, 2026by deng451e
AI Summary
This is a major update to LMCache introducing a robust Multi-Process (MP) mode, Hybrid Memory Allocator (HMA) support, and new storage backends. It includes significant architectural changes to the configuration interface and adds SHM-based data transfer paths for improved performance across GPUs, CPUs, and accelerators.
Key Highlights
- Introduction of Multi-Process (MP) coordinator backbone and server registration
- Support for Hybrid Memory Allocator (HMA) and Mamba/GDN hybrid models (Qwen3.5)
- New SHM-based data transfer path and POSIX SHM infra for CPU KV-cache IPC
- Enhanced CacheBlend v3 with token-level matching for non-block-aligned KV reuse
- New storage backends including NVIDIA CMX (NIXL), Cloud Bigtable, and Moore Threads MUSA
Breaking Changes
- LMCacheGroupView renamed to EngineGroupInfo
- report_status is now per-kernel-group
- Per-group tokens_per_chunk and slots_per_chunk now used instead of inferring from cache_config.block_size
- goblin is deprecated
- python_ops_fallback now requires completion recorder ops
New Features
- MP coordinator backbone and CLI
- Hybrid Memory Allocator (HMA) support
- NIXL DOCA_MEMOS storage backend
- Cloud Bigtable remote storage connector
- Moore Threads MUSA support
- Token-level matching for CacheBlend
Full Release Notes
## LMCache v0.4.7 Release ## Interface / Config / CLI / Build Changes **Breaking / behavior changes (action may be needed)** - `python_ops_fallback` now requires completion recorder ops (added missing ops) - `LMCacheGroupView` renamed to `EngineGroupInfo` - `report_status` is now per-kernel-group - Per-group `tokens_per_chunk` / `slots_per_chunk` now used instead of inferring from `cache_config.block_size` - `goblin` is deprecated (documented) - Blend v2 CI removed; CacheBlend now uses Blend v3 **New / additive (opt-in)** - New `mp_transfer_mode` config option - New SHM-based data transfer path for GPUs/CPU/Accelerators (POSIX SHM infra for CPU KV-cache IPC) - New hybrid memory allocator (HMA) support, with per-group block sizes and Mamba/GDN hybrid model (Qwen3.5) support - New MP coordinator backbone: server registration, coordinator CLI, L2 quota/usage/eviction, global CacheBlend fingerprint directory - New CLI quota management commands (set/get/list/delete) - New runtime DAX hotplug HTTP API (MP) - New `--mode cpu` and `--transfer-mode` options in `server_bench` - New backends: NIXL DOCA_MEMOS (NVIDIA CMX), Cloud Bigtable remote storage, Moore Threads MUSA support, multipath KV-cache offloading in NIXL backend - New `multi_layer_block_kv_transfer` unified MP transfer primitive - LMCache startup banner now printed in CLI and vLLM connectors - vLLM CPU 2-fused KV layout support - Token-level matching for non-block-aligned KV reuse (CacheBlend) ### MP (Multi-Process Mode) - #3245 Retain CUDA IPC events in MP adapter - #3359 SHM-based data transfer path for GPUs/CPU/Accelerators - #3382 Fix GPU block exhaustion deadlock at high concurrency with chunked KV loading - #3488 Add mp coordinator backbone - #3513 Add mp_transfer_mode config option - #3516 Register MP servers with the coordinator - #3522 Add coordinator CLI and mp server registration - #3531 Introduce create_cache_context factory - #3557 Refactor LMCache layer group for better compat with hybrid models - #3608 Introduce object_group_id into the ObjectKey - #3352 Add SHM-based NonGpuContext (server-side copy) - #3612 Implement interface for multi-object group and sliding window support (HMA) - #3630 Coordinator L2 Quota, Usage, Eviction - #3597 Global CacheBlend fingerprint directory on the MP coordinator - #3264 Add runtime DAX hotplug http API - #3477 Add l2_evicted_object, add cachesalt to L1/L2 metrics - #3478 Consolidate ParallelStrategy construction in vllm_multi_process_adapter - #3558 Align MP server id with OTel service.instance.id - #3508 Add multi_layer_block_kv_transfer Python fallback as unified MP transfer primitive - #3563 Add POSIX SHM infra for CPU KV-cache IPC ### Core / HMA - #3419 Add support for hybrid memory allocator - #3491 Bitmap-based prefetch result + pluggable TrimPolicy - #3503 Native bulk-set: build found bitmap via batched_set + gather - #3492 Sparse prefetch via TrimPolicy.SPARSE + covered_keys - #3521 Support different block size for different groups - #3613 Support Mamba/GDN hybrid models (Qwen3.5) - #3616 Per-group tokens_per_chunk and slots_per_chunk - #3635 Optimize DSV4 store/load size - #3589 Add GDS L1 slab-file tier (cuFile DMA) for MP mode ### CacheBlend - #3364 Blend v3 - #3582 Token-level matching + per-token slot scatter for non-block-aligned KV reuse - #3629 Reuse gpu_transfer.cache_contexts; drop CB GPU-context mirror - #3541 Cleanup/remove blend v2 ci ### Storage / Backends - #3486 NIXL DOCA_MEMOS storage backend (NVIDIA CMX) - #3453 nixl_storage: use LocalCPUBackend if nixl_buffer_device=cpu - #3263 Added HFbucket MP - #2418 Add multipath KV-cache offloading support in LMCache NIXL backend - #3404 Integrate native Cloud Bigtable remote storage connector - #3483 Add Moore Threads MUSA support for LMCache v1 - #3568 nixl: create storage directory if it doesn't exist - #3274 Missing io_uring changes + nvme io_uring_cmd passthrough ### Observability - #3384 Add NVTX annotations to LocalDiskBackend disk read path - #3607 Blend server trace sub-spans + V3 hit-rate breakdown ### Operator - #3543 CacheBlend: CacheBlendEngine CRD + injection webhook - #3647 Emit --engine-type blend for CacheBlend engine - #3646 Install cert-manager in e2e smoke suite ### CLI - #3611 Print LMCache startup banner in CLI and vLLM connectors - #3625 Refactor query and trace cli - #3623 Add quota management commands (set/get/list/delete) ### XPU / Accelerators - #3360 Add SYCL CacheGen + RoPE kernels and in-process blender XPU tests ### Bugfixes - #3327 gds: use parse_cache_key to handle LayerCacheEngineKey on restart - #3441 Drop EngineArgs+asdict to fix vLLM 0.20+ pydantic error - #3189 Fix LocalCPUBackend recovery when pinned CPU chunks block eviction - #3469 Add missing completion recorder ops to python_ops_fallback - #3463 Prevent stale prefetches and registry memory leaks by purging unregistered KV layouts - #3410 Prevent negative pin count on unpinned remote memory objects - #3278 PD restore pin=True in PD sync backend dedup path - #3525 Resolve AttributeError in test_execute_calls_run_http_server - #3602 Handle NL_X_NB_NH_BS_TWO_HS in get_group_data_ptrs - #3606 Add missing enum to GPUVKFormat - #3325 Graceful skip on slot_mapping/token_ids desync in wait_for_save (fixes #3318) - #3648 Correct retrieve log label prefix -> non_shifted ### Performance / Optimization - #3413 Avoid redundant PCIe transfer on leader rank during retrieve - #3591 Optimize Python fallback path for block transfer operations ### Refactor / Cleanup - #3460 Move serializer registry + encoder/decoder helpers to end of custom_types.py - #3445 Simplify redundant conditions in RawBlockCore - #3216 Put lmcache_frontend into lmcache repo - #3514 Add set_shape_desc_dtype helper to avoid scattered try/except - #3545 Normalize block_ids to tolerate legacy vLLM connectors - #3577 Normalize flat/nested block_ids in flat_block_ids and connector __str__ - #3567 Support vLLM CPU 2-fused KV layout - #3598 Rename LMCacheGroupView to EngineGroupInfo - #3599 Change report_status to be per-kernel-group in LMCache - #3581 Remove unnecessary global statement in cuda_extension - #3600 Utilize multi_layer_block_kv_transfer ops for data transfer path - #3524 Add transfer timing logs to non-GPU path similar to CUDA path ### Benchmarking - #3283 Support benchmark fs and hf3fs backend via storage_backend_io_benchmark - #3528 server_bench supports --mode cpu and --transfer-mode - #3603 Support aligned L1 buffers for L2 adapters ### CI/CD & Build - #3456 Add http_api e2e test for MP HTTP server endpoints and CLI commands - #3498 Relax timeout to reduce flakiness of some CI/CD tests - #3502 Force vLLM Model Runner V1 in the PD comprehensive test - #3489 Add pickle/shm vLLM + LMCache e2e validation on CPU - #3538 Hot fix for the CPU test in multiprocess mode CI - #3507 Add parity test between c_ops and python_ops_fallback - #3321 Add unit tests for v1/utils/bloom_filter - #3556 Improve CI stability: gemma-4 test & serde test - #3614 Reduce ci cpu e2e test memory request - #3621 cu129 images: pin vllm to the cu129 index (drop unsafe-best-match) - #3590 Add CPU e2e test (vLLM and bench server) ### Docs - #3457 Update and restructure CLI reference - #3481 Fix Docker examples and build metadata - #3501 Combined doc drift updates May 27-Jun 2 - #3504 KV Cache Size Calculator: add hybrid SWA, DSA, placeholders for Mamba / Linear - #3506 Add recipe for Gemma 3 - #3518 Deprecate goblin in doc - #3461 Update README.md - #3433 Auto-select model in CPU-offloading example to fit GPU - #3534 Add filesystem connector backend guide - #3645 Recipe update for Qwen 3.6 27B and general guideline for mamba models - #2834 kv_cache_calculator: add Hunyuan & DeepSeek models, fix head_dim/CLA, add i18n UI ### Chinese Translation - #3386 Update Chinese documentation translations - #3482 Update Chinese documentation translations - #3588 Update Chinese documentation translations - #3592 Correct machine translation errors in documentation ### Chore / Maintenance - #3443 Convert loglevel_api f-strings to %-format - #3447 Convert internal_api_server f-string log calls to %-format - #3136 Bump go.opentelemetry.io/otel from 1.36.0 to 1.41.0 in /operator - #3596 Bump sphinxcontrib-mermaid from 1.2.2 to 2.0.2 ## New Contributors - @ChiragB254 made their first contribution in #3443 - @Alorun made their first contribution in #3447 - @JinuJeong made their first contribution in #3327 - @catyion made their first contribution in #3460 - @3xdevv made their first contribution in #3481 - @nayeonikim made their first contribution in #3445 - @sihara made their first contribution in #3384 - @XuanCS made their first contribution in #3278 - @Lyj1007 made their first contribution in #3507 - @kirklandsign made their first contribution in #3321 - @superleo made their first contribution in #3483 - @feixiangpeng made their first contribution in #3263 - @sonimwang made their first contribution in #3592 - @KimmoZAG made their first contribution in #2834 - @Chris-Sigopt made their first contribution in #3606 - @ekaynar made their first contribution in #2418 - @dhruvatr made their first contribution in #3581 - @Kushagra963-lab made their first contribution in #3534