v0.4.7

microsoft/markitdownv0.4.7Jun 13, 2026by deng451e

AI Summary

This is a major update to LMCache introducing a robust Multi-Process (MP) mode, Hybrid Memory Allocator (HMA) support, and new storage backends. It includes significant architectural changes to the configuration interface and adds SHM-based data transfer paths for improved performance across GPUs, CPUs, and accelerators.

Key Highlights

  • Introduction of Multi-Process (MP) coordinator backbone and server registration
  • Support for Hybrid Memory Allocator (HMA) and Mamba/GDN hybrid models (Qwen3.5)
  • New SHM-based data transfer path and POSIX SHM infra for CPU KV-cache IPC
  • Enhanced CacheBlend v3 with token-level matching for non-block-aligned KV reuse
  • New storage backends including NVIDIA CMX (NIXL), Cloud Bigtable, and Moore Threads MUSA

Breaking Changes

  • LMCacheGroupView renamed to EngineGroupInfo
  • report_status is now per-kernel-group
  • Per-group tokens_per_chunk and slots_per_chunk now used instead of inferring from cache_config.block_size
  • goblin is deprecated
  • python_ops_fallback now requires completion recorder ops

New Features

  • MP coordinator backbone and CLI
  • Hybrid Memory Allocator (HMA) support
  • NIXL DOCA_MEMOS storage backend
  • Cloud Bigtable remote storage connector
  • Moore Threads MUSA support
  • Token-level matching for CacheBlend

Full Release Notes

## LMCache v0.4.7 Release

## Interface / Config / CLI / Build Changes

**Breaking / behavior changes (action may be needed)**
- `python_ops_fallback` now requires completion recorder ops (added missing ops)
- `LMCacheGroupView` renamed to `EngineGroupInfo`
- `report_status` is now per-kernel-group
- Per-group `tokens_per_chunk` / `slots_per_chunk` now used instead of inferring from `cache_config.block_size`
- `goblin` is deprecated (documented)
- Blend v2 CI removed; CacheBlend now uses Blend v3

**New / additive (opt-in)**
- New `mp_transfer_mode` config option
- New SHM-based data transfer path for GPUs/CPU/Accelerators (POSIX SHM infra for CPU KV-cache IPC)
- New hybrid memory allocator (HMA) support, with per-group block sizes and Mamba/GDN hybrid model (Qwen3.5) support
- New MP coordinator backbone: server registration, coordinator CLI, L2 quota/usage/eviction, global CacheBlend fingerprint directory
- New CLI quota management commands (set/get/list/delete)
- New runtime DAX hotplug HTTP API (MP)
- New `--mode cpu` and `--transfer-mode` options in `server_bench`
- New backends: NIXL DOCA_MEMOS (NVIDIA CMX), Cloud Bigtable remote storage, Moore Threads MUSA support, multipath KV-cache offloading in NIXL backend
- New `multi_layer_block_kv_transfer` unified MP transfer primitive
- LMCache startup banner now printed in CLI and vLLM connectors
- vLLM CPU 2-fused KV layout support
- Token-level matching for non-block-aligned KV reuse (CacheBlend)

### MP (Multi-Process Mode)
- #3245 Retain CUDA IPC events in MP adapter
- #3359 SHM-based data transfer path for GPUs/CPU/Accelerators
- #3382 Fix GPU block exhaustion deadlock at high concurrency with chunked KV loading
- #3488 Add mp coordinator backbone
- #3513 Add mp_transfer_mode config option
- #3516 Register MP servers with the coordinator
- #3522 Add coordinator CLI and mp server registration
- #3531 Introduce create_cache_context factory
- #3557 Refactor LMCache layer group for better compat with hybrid models
- #3608 Introduce object_group_id into the ObjectKey
- #3352 Add SHM-based NonGpuContext (server-side copy)
- #3612 Implement interface for multi-object group and sliding window support (HMA)
- #3630 Coordinator L2 Quota, Usage, Eviction
- #3597 Global CacheBlend fingerprint directory on the MP coordinator
- #3264 Add runtime DAX hotplug http API
- #3477 Add l2_evicted_object, add cachesalt to L1/L2 metrics
- #3478 Consolidate ParallelStrategy construction in vllm_multi_process_adapter
- #3558 Align MP server id with OTel service.instance.id
- #3508 Add multi_layer_block_kv_transfer Python fallback as unified MP transfer primitive
- #3563 Add POSIX SHM infra for CPU KV-cache IPC

### Core / HMA
- #3419 Add support for hybrid memory allocator
- #3491 Bitmap-based prefetch result + pluggable TrimPolicy
- #3503 Native bulk-set: build found bitmap via batched_set + gather
- #3492 Sparse prefetch via TrimPolicy.SPARSE + covered_keys
- #3521 Support different block size for different groups
- #3613 Support Mamba/GDN hybrid models (Qwen3.5)
- #3616 Per-group tokens_per_chunk and slots_per_chunk
- #3635 Optimize DSV4 store/load size
- #3589 Add GDS L1 slab-file tier (cuFile DMA) for MP mode

### CacheBlend  
- #3364 Blend v3
- #3582 Token-level matching + per-token slot scatter for non-block-aligned KV reuse
- #3629 Reuse gpu_transfer.cache_contexts; drop CB GPU-context mirror
- #3541 Cleanup/remove blend v2 ci

### Storage / Backends
- #3486 NIXL DOCA_MEMOS storage backend (NVIDIA CMX)
- #3453 nixl_storage: use LocalCPUBackend if nixl_buffer_device=cpu
- #3263 Added HFbucket MP
- #2418 Add multipath KV-cache offloading support in LMCache NIXL backend
- #3404 Integrate native Cloud Bigtable remote storage connector
- #3483 Add Moore Threads MUSA support for LMCache v1
- #3568 nixl: create storage directory if it doesn't exist
- #3274 Missing io_uring changes + nvme io_uring_cmd passthrough

### Observability
- #3384 Add NVTX annotations to LocalDiskBackend disk read path
- #3607 Blend server trace sub-spans + V3 hit-rate breakdown

### Operator
- #3543 CacheBlend: CacheBlendEngine CRD + injection webhook
- #3647 Emit --engine-type blend for CacheBlend engine
- #3646 Install cert-manager in e2e smoke suite

### CLI
- #3611 Print LMCache startup banner in CLI and vLLM connectors
- #3625 Refactor query and trace cli
- #3623 Add quota management commands (set/get/list/delete)

### XPU / Accelerators
- #3360 Add SYCL CacheGen + RoPE kernels and in-process blender XPU tests

### Bugfixes
- #3327 gds: use parse_cache_key to handle LayerCacheEngineKey on restart
- #3441 Drop EngineArgs+asdict to fix vLLM 0.20+ pydantic error
- #3189 Fix LocalCPUBackend recovery when pinned CPU chunks block eviction
- #3469 Add missing completion recorder ops to python_ops_fallback
- #3463 Prevent stale prefetches and registry memory leaks by purging unregistered KV layouts
- #3410 Prevent negative pin count on unpinned remote memory objects
- #3278 PD restore pin=True in PD sync backend dedup path
- #3525 Resolve AttributeError in test_execute_calls_run_http_server
- #3602 Handle NL_X_NB_NH_BS_TWO_HS in get_group_data_ptrs
- #3606 Add missing enum to GPUVKFormat
- #3325 Graceful skip on slot_mapping/token_ids desync in wait_for_save (fixes #3318)
- #3648 Correct retrieve log label prefix -> non_shifted

### Performance / Optimization
- #3413 Avoid redundant PCIe transfer on leader rank during retrieve
- #3591 Optimize Python fallback path for block transfer operations

### Refactor / Cleanup
- #3460 Move serializer registry + encoder/decoder helpers to end of custom_types.py
- #3445 Simplify redundant conditions in RawBlockCore
- #3216 Put lmcache_frontend into lmcache repo
- #3514 Add set_shape_desc_dtype helper to avoid scattered try/except
- #3545 Normalize block_ids to tolerate legacy vLLM connectors
- #3577 Normalize flat/nested block_ids in flat_block_ids and connector __str__
- #3567 Support vLLM CPU 2-fused KV layout
- #3598 Rename LMCacheGroupView to EngineGroupInfo
- #3599 Change report_status to be per-kernel-group in LMCache
- #3581 Remove unnecessary global statement in cuda_extension
- #3600 Utilize multi_layer_block_kv_transfer ops for data transfer path
- #3524 Add transfer timing logs to non-GPU path similar to CUDA path

### Benchmarking
- #3283 Support benchmark fs and hf3fs backend via storage_backend_io_benchmark
- #3528 server_bench supports --mode cpu and --transfer-mode
- #3603 Support aligned L1 buffers for L2 adapters

### CI/CD & Build
- #3456 Add http_api e2e test for MP HTTP server endpoints and CLI commands
- #3498 Relax timeout to reduce flakiness of some CI/CD tests
- #3502 Force vLLM Model Runner V1 in the PD comprehensive test
- #3489 Add pickle/shm vLLM + LMCache e2e validation on CPU
- #3538 Hot fix for the CPU test in multiprocess mode CI
- #3507 Add parity test between c_ops and python_ops_fallback
- #3321 Add unit tests for v1/utils/bloom_filter
- #3556 Improve CI stability: gemma-4 test & serde test
- #3614 Reduce ci cpu e2e test memory request
- #3621 cu129 images: pin vllm to the cu129 index (drop unsafe-best-match)
- #3590 Add CPU e2e test (vLLM and bench server)

### Docs
- #3457 Update and restructure CLI reference
- #3481 Fix Docker examples and build metadata
- #3501 Combined doc drift updates May 27-Jun 2
- #3504 KV Cache Size Calculator: add hybrid SWA, DSA, placeholders for Mamba / Linear
- #3506 Add recipe for Gemma 3
- #3518 Deprecate goblin in doc
- #3461 Update README.md
- #3433 Auto-select model in CPU-offloading example to fit GPU
- #3534 Add filesystem connector backend guide
- #3645 Recipe update for Qwen 3.6 27B and general guideline for mamba models
- #2834 kv_cache_calculator: add Hunyuan & DeepSeek models, fix head_dim/CLA, add i18n UI

### Chinese Translation
- #3386 Update Chinese documentation translations
- #3482 Update Chinese documentation translations
- #3588 Update Chinese documentation translations
- #3592 Correct machine translation errors in documentation

### Chore / Maintenance
- #3443 Convert loglevel_api f-strings to %-format
- #3447 Convert internal_api_server f-string log calls to %-format
- #3136 Bump go.opentelemetry.io/otel from 1.36.0 to 1.41.0 in /operator
- #3596 Bump sphinxcontrib-mermaid from 1.2.2 to 2.0.2

## New Contributors
- @ChiragB254 made their first contribution in #3443
- @Alorun made their first contribution in #3447
- @JinuJeong made their first contribution in #3327
- @catyion made their first contribution in #3460
- @3xdevv made their first contribution in #3481
- @nayeonikim made their first contribution in #3445
- @sihara made their first contribution in #3384
- @XuanCS made their first contribution in #3278
- @Lyj1007 made their first contribution in #3507
- @kirklandsign made their first contribution in #3321
- @superleo made their first contribution in #3483
- @feixiangpeng made their first contribution in #3263
- @sonimwang made their first contribution in #3592
- @KimmoZAG made their first contribution in #2834
- @Chris-Sigopt made their first contribution in #3606
- @ekaynar made their first contribution in #2418
- @dhruvatr made their first contribution in #3581
- @Kushagra963-lab made their first contribution in #3534