Starstruk Logo MarkStarstruk

v0.55.0

Mesh-LLM/mesh-llmv0.55.0Apr 3, 2026by github-actions[bot]

AI Summary

Focuses on asymmetric KV caching for larger models, runtime model management, and critical connection stability fixes.

Key Highlights

  • Implemented asymmetric KV cache (Q8_0 K + Q4_0 V) for 5-50GB models
  • Added runtime model load/unload via CLI and REST
  • Fixed critical client --auto join hang (connect_to_peer regression)
  • Added context-aware host selection and retry logic

New Features

  • Asymmetric KV cache
  • Runtime model load/unload
  • Context-aware routing with retry logic

Full Release Notes

## What's Changed
* feature: add support to skip jobs that don't have modified files in CI by @ndizazzo in https://github.com/michaelneale/mesh-llm/pull/116
* launch: asymmetric KV cache — Q8_0 K + Q4_0 V for 5-50GB models by @michaelneale in https://github.com/michaelneale/mesh-llm/pull/104
* Fix compilation errors from missing `served_model_descriptors` in test struct initializers by @Copilot in https://github.com/michaelneale/mesh-llm/pull/120
* Fix compile errors in model descriptor test helpers by @Copilot in https://github.com/michaelneale/mesh-llm/pull/121
* Fix UI build to fail on TypeScript compile errors by @i386 in https://github.com/michaelneale/mesh-llm/pull/124
* Add runtime model load/unload with CLI and REST by @i386 in https://github.com/michaelneale/mesh-llm/pull/56
* URGENT: Fix client --auto join hang (connect_to_peer regression) by @michaelneale in https://github.com/michaelneale/mesh-llm/pull/126
* ci: add client-auto boot test by @michaelneale in https://github.com/michaelneale/mesh-llm/pull/125
* Add canonical served model descriptors by @i386 in https://github.com/michaelneale/mesh-llm/pull/118
* Decomposing the codebase by @i386 in https://github.com/michaelneale/mesh-llm/pull/127
* Remove Windows builds from CI by @i386 in https://github.com/michaelneale/mesh-llm/pull/140
* router: context-aware host selection and retry on context overflow by @michaelneale in https://github.com/michaelneale/mesh-llm/pull/141


**Full Changelog**: https://github.com/michaelneale/mesh-llm/compare/v0.54.0...v0.55.0