v0.5.1

# LMCache v0.5.1 Release Notes

## Highlights: Multi-Process (MP) mode maturity & memory-tier expansion

- **Warm-prefetch and tiered caching improvements** — the MP coordinator can now warm-prefetch KV from L2 into L1 ([[#3827](https://github.com/LMCache/LMCache/pull/3827)](https://github.com/LMCache/LMCache/pull/3827)), Device-DAX L1 is split cleanly from the CPU L1 manager ([[#3947](https://github.com/LMCache/LMCache/pull/3947)](https://github.com/LMCache/LMCache/pull/3947)), and a new `HiddenStateStore` enables hidden-state caching ([[#3221](https://github.com/LMCache/LMCache/pull/3221)](https://github.com/LMCache/LMCache/pull/3221)).
- **Broader hardware backends** — added an AMD `hipFile` backend for the GDS L1 tier in MP mode ([[#3843](https://github.com/LMCache/LMCache/pull/3843)](https://github.com/LMCache/LMCache/pull/3843)), a guarded MUSA MP handle wrapper ([[#3756](https://github.com/LMCache/LMCache/pull/3756)](https://github.com/LMCache/LMCache/pull/3756)), and an XPU Docker image ([[#3838](https://github.com/LMCache/LMCache/pull/3838)](https://github.com/LMCache/LMCache/pull/3838)).
- **Platform abstraction** — auto-discovery of `DeviceIPCWrapper` subclasses under `platform/<device>/` ([[#3829](https://github.com/LMCache/LMCache/pull/3829)](https://github.com/LMCache/LMCache/pull/3829)) and pin-memory abstracted behind `DeviceExt` with pinned SHM for async D2H ([[#3823](https://github.com/LMCache/LMCache/pull/3823)](https://github.com/LMCache/LMCache/pull/3823)).
- **Model support** — MiniMax M3 layer grouping via `EngineKVFormat` ([[#3779](https://github.com/LMCache/LMCache/pull/3779)](https://github.com/LMCache/LMCache/pull/3779)), with all grouping queries consolidated onto `EngineKVFormat` (also fixing SGLang tests) ([[#3907](https://github.com/LMCache/LMCache/pull/3907)](https://github.com/LMCache/LMCache/pull/3907)).
- **Operator automation** — auto-inject `LMCacheEngine` connection into vLLM pods via webhook ([[#3822](https://github.com/LMCache/LMCache/pull/3822)](https://github.com/LMCache/LMCache/pull/3822)), configurable `hostNetwork` on the CRD ([[#3849](https://github.com/LMCache/LMCache/pull/3849)](https://github.com/LMCache/LMCache/pull/3849)), and optional privileged-mode DaemonSet ([[#3943](https://github.com/LMCache/LMCache/pull/3943)](https://github.com/LMCache/LMCache/pull/3943)).

---

## Features

- `HiddenStateStore` for hidden-state caching ([[#3221](https://github.com/LMCache/LMCache/pull/3221)](https://github.com/LMCache/LMCache/pull/3221))
- TurboQuant serde support for L2 adapters ([[#3193](https://github.com/LMCache/LMCache/pull/3193)](https://github.com/LMCache/LMCache/pull/3193))
- Token dropping SDK v3 ([[#3820](https://github.com/LMCache/LMCache/pull/3820)](https://github.com/LMCache/LMCache/pull/3820))
- Per-key TTL + partial-chunk validation for the Valkey glide connector ([[#3836](https://github.com/LMCache/LMCache/pull/3836)](https://github.com/LMCache/LMCache/pull/3836))
- Configurable `disk_io_threads` for local disk and GDS backends ([[#3941](https://github.com/LMCache/LMCache/pull/3941)](https://github.com/LMCache/LMCache/pull/3941))
- Session-expiry cleanup thread for `SessionManager` ([[#3570](https://github.com/LMCache/LMCache/pull/3570)](https://github.com/LMCache/LMCache/pull/3570))
- Propagate sliding-window info to the prefetch path ([[#3776](https://github.com/LMCache/LMCache/pull/3776)](https://github.com/LMCache/LMCache/pull/3776))

## Multi-Process (MP) / Memory Tiers

- Warm-prefetch KV from L2 into L1 in the MP coordinator ([[#3827](https://github.com/LMCache/LMCache/pull/3827)](https://github.com/LMCache/LMCache/pull/3827))
- Split Device-DAX L1 from the CPU L1 manager ([[#3947](https://github.com/LMCache/LMCache/pull/3947)](https://github.com/LMCache/LMCache/pull/3947))
- AMD `hipFile` backend for the GDS L1 tier (MP mode) ([[#3843](https://github.com/LMCache/LMCache/pull/3843)](https://github.com/LMCache/LMCache/pull/3843))
- Guarded MUSA MP handle wrapper ([[#3756](https://github.com/LMCache/LMCache/pull/3756)](https://github.com/LMCache/LMCache/pull/3756))
- Group MP server & coordinator HTTP APIs into endpoint groups ([[#3948](https://github.com/LMCache/LMCache/pull/3948)](https://github.com/LMCache/LMCache/pull/3948))

## Platform / Hardware

- Auto-discover `DeviceIPCWrapper` subclasses under `platform/<device>/` ([[#3829](https://github.com/LMCache/LMCache/pull/3829)](https://github.com/LMCache/LMCache/pull/3829))
- Abstract pin memory behind `DeviceExt` and pin SHM for async D2H ([[#3823](https://github.com/LMCache/LMCache/pull/3823)](https://github.com/LMCache/LMCache/pull/3823))
- Add LMCache XPU Docker file ([[#3838](https://github.com/LMCache/LMCache/pull/3838)](https://github.com/LMCache/LMCache/pull/3838))

## Model Support

- Group layers with `EngineKVFormat` for MiniMax M3 ([[#3779](https://github.com/LMCache/LMCache/pull/3779)](https://github.com/LMCache/LMCache/pull/3779))
- Consolidate all grouping queries on `EngineKVFormat` (also fixes SGLang tests) ([[#3907](https://github.com/LMCache/LMCache/pull/3907)](https://github.com/LMCache/LMCache/pull/3907))

## Operator

- Auto-inject `LMCacheEngine` connection into vLLM pods via webhook ([[#3822](https://github.com/LMCache/LMCache/pull/3822)](https://github.com/LMCache/LMCache/pull/3822))
- Add configurable `hostNetwork` field to `LMCacheEngine` CRD ([[#3849](https://github.com/LMCache/LMCache/pull/3849)](https://github.com/LMCache/LMCache/pull/3849))
- Make engine DaemonSet privileged mode optional ([[#3943](https://github.com/LMCache/LMCache/pull/3943)](https://github.com/LMCache/LMCache/pull/3943))

## Bugfixes

- `LMCacheMPConnector` crashes under `MultiConnector` in PD setups ([[#3866](https://github.com/LMCache/LMCache/pull/3866)](https://github.com/LMCache/LMCache/pull/3866))
- Fix missing `cache_salt` in `free_lookup_locks` call (MP) ([[#3771](https://github.com/LMCache/LMCache/pull/3771)](https://github.com/LMCache/LMCache/pull/3771))
- Fix P2P coordinator server disconnected error ([[#3876](https://github.com/LMCache/LMCache/pull/3876)](https://github.com/LMCache/LMCache/pull/3876))
- Graceful skip on Scheduler-role abort in `request_finished` ([[#3340](https://github.com/LMCache/LMCache/pull/3340)](https://github.com/LMCache/LMCache/pull/3340))
- No-op KV transfer when LMCache is in degraded mode (vllm/v1) ([[#3487](https://github.com/LMCache/LMCache/pull/3487)](https://github.com/LMCache/LMCache/pull/3487))
- Clean up local backend lock handling ([[#3682](https://github.com/LMCache/LMCache/pull/3682)](https://github.com/LMCache/LMCache/pull/3682))
- Roll back put-task refs when RawBlock dispatch scheduling fails ([[#3698](https://github.com/LMCache/LMCache/pull/3698)](https://github.com/LMCache/LMCache/pull/3698))
- Allow non-zero `storage_offset` in dim-0-padded layout (MTP + CPU offload) ([[#3853](https://github.com/LMCache/LMCache/pull/3853)](https://github.com/LMCache/LMCache/pull/3853))
- Cache ctypes ubyte-array types to stop heap-type leak ([[#3793](https://github.com/LMCache/LMCache/pull/3793)](https://github.com/LMCache/LMCache/pull/3793))
- `batched_nixl_desc_exists` never hits on FILE (dynamic NIXL) ([[#3520](https://github.com/LMCache/LMCache/pull/3520)](https://github.com/LMCache/LMCache/pull/3520))
- Pin nixl backend directly so only one CUDA variant installs ([[#3883](https://github.com/LMCache/LMCache/pull/3883)](https://github.com/LMCache/LMCache/pull/3883))
- Fix dirty CMX read in static backend ([[#3937](https://github.com/LMCache/LMCache/pull/3937)](https://github.com/LMCache/LMCache/pull/3937))
- Fix `_alloc_page_aligned_pinned_view()` ([[#3839](https://github.com/LMCache/LMCache/pull/3839)](https://github.com/LMCache/LMCache/pull/3839))
- Fix unit tests ([[#3878](https://github.com/LMCache/LMCache/pull/3878)](https://github.com/LMCache/LMCache/pull/3878))

## Refactoring

- Route affinity thread pool on dense worker rank instead of hashed identity ([[#3905](https://github.com/LMCache/LMCache/pull/3905)](https://github.com/LMCache/LMCache/pull/3905))
- Move type-only imports under `TYPE_CHECKING` — l2_adapters ([[#3736](https://github.com/LMCache/LMCache/pull/3736)](https://github.com/LMCache/LMCache/pull/3736)), `IPCCacheServerKey` ([[#3800](https://github.com/LMCache/LMCache/pull/3800)](https://github.com/LMCache/LMCache/pull/3800)), and general cleanup ([[#3871](https://github.com/LMCache/LMCache/pull/3871)](https://github.com/LMCache/LMCache/pull/3871))
- Convert f-string log calls to `%`-format: `cache_engine.py` ([[#3784](https://github.com/LMCache/LMCache/pull/3784)](https://github.com/LMCache/LMCache/pull/3784)), `system_detection.py` ([[#3797](https://github.com/LMCache/LMCache/pull/3797)](https://github.com/LMCache/LMCache/pull/3797), [[#3795](https://github.com/LMCache/LMCache/pull/3795)](https://github.com/LMCache/LMCache/pull/3795)), `storage_backend` ([[#3975](https://github.com/LMCache/LMCache/pull/3975)](https://github.com/LMCache/LMCache/pull/3975)), `s3_connector.py` ([[#3977](https://github.com/LMCache/LMCache/pull/3977)](https://github.com/LMCache/LMCache/pull/3977)), `token_database` ([[#3982](https://github.com/LMCache/LMCache/pull/3982)](https://github.com/LMCache/LMCache/pull/3982))

## Performance

- Collapse object-group KV transfer into one GIL-released native call ([[#3908](https://github.com/LMCache/LMCache/pull/3908)](https://github.com/LMCache/LMCache/pull/3908))

## CLI / Tooling

- Output the model answer in `lmcache query engine` ([[#3887](https://github.com/LMCache/LMCache/pull/3887)](https://github.com/LMCache/LMCache/pull/3887))
- Add `describe engine` command ([[#3867](https://github.com/LMCache/LMCache/pull/3867)](https://github.com/LMCache/LMCache/pull/3867))
- `bench engine`: add back navigation, exit, and URL shorthand to interactive setup ([[#3953](https://github.com/LMCache/LMCache/pull/3953)](https://github.com/LMCache/LMCache/pull/3953))
- Allow bare host in `lmcache.mp.host` ([[#3951](https://github.com/LMCache/LMCache/pull/3951)](https://github.com/LMCache/LMCache/pull/3951))

## CI / Build

- cu129 release image: use stable vLLM wheel instead of nightly ([[#3946](https://github.com/LMCache/LMCache/pull/3946)](https://github.com/LMCache/LMCache/pull/3946))
- Add CUDA engine-driven MP mode for gsm8k to prevent functional and accuracy regressions ([[#3860](https://github.com/LMCache/LMCache/pull/3860)](https://github.com/LMCache/LMCache/pull/3860))
- Pin vLLM nightly to the canary-verified version ([[#3910](https://github.com/LMCache/LMCache/pull/3910)](https://github.com/LMCache/LMCache/pull/3910))
- Support timer-trigger test vllm and record ([[#3920](https://github.com/LMCache/LMCache/pull/3920)](https://github.com/LMCache/LMCache/pull/3920))
- Retry flaky multiprocess tests up to 3 times ([[#3816](https://github.com/LMCache/LMCache/pull/3816)](https://github.com/LMCache/LMCache/pull/3816))

## Docs

- Polish README update wording ([[#3542](https://github.com/LMCache/LMCache/pull/3542)](https://github.com/LMCache/LMCache/pull/3542))
- Remove duplicate Extension Guide tab ([[#3850](https://github.com/LMCache/LMCache/pull/3850)](https://github.com/LMCache/LMCache/pull/3850))
- Fix CRD spec docs and policy enum gaps ([[#3496](https://github.com/LMCache/LMCache/pull/3496)](https://github.com/LMCache/LMCache/pull/3496))
- Update developer guide CLI docs to match the auto-discovery framework ([[#3851](https://github.com/LMCache/LMCache/pull/3851)](https://github.com/LMCache/LMCache/pull/3851))
- Update CLI docs ([[#3870](https://github.com/LMCache/LMCache/pull/3870)](https://github.com/LMCache/LMCache/pull/3870))
- Fix MP observability `/metrics` port and add a Grafana guide ([[#3962](https://github.com/LMCache/LMCache/pull/3962)](https://github.com/LMCache/LMCache/pull/3962))
- Add deprecation warning for non-MP docs ([[#3963](https://github.com/LMCache/LMCache/pull/3963)](https://github.com/LMCache/LMCache/pull/3963))
- Consolidate daily drift-check updates (2026-06-29) ([[#3944](https://github.com/LMCache/LMCache/pull/3944)](https://github.com/LMCache/LMCache/pull/3944))
- Update Chinese documentation translations ([[#3811](https://github.com/LMCache/LMCache/pull/3811)](https://github.com/LMCache/LMCache/pull/3811)), fix translation & update welcome page ([[#3906](https://github.com/LMCache/LMCache/pull/3906)](https://github.com/LMCache/LMCache/pull/3906)), and fix translation reuse job ([[#3890](https://github.com/LMCache/LMCache/pull/3890)](https://github.com/LMCache/LMCache/pull/3890))

## Observability

- Raise timeout event upon timeout error ([[#3964](https://github.com/LMCache/LMCache/pull/3964)](https://github.com/LMCache/LMCache/pull/3964))

## New Contributors

@allytotheson, @chfeng-cs, @shihaoustc, @wsyjh8, @nevasini1, @sat-v4, @wangzhehan0811, @THINKER-ONLY, @a-m-n-s, @nafis271, @iyastreb, @Anai-Guo, @kartikhans, @waynel96 — thank you!

**Full Changelog**: https://github.com/LMCache/LMCache/compare/v0.5.0...v0.5.1