@magnitudedev/cli@0.2.4 @magnitudedev/cli@0.2.5
immich-app/immichOct 2, 2026 → Oct 3, 20261 day between releases
What's New
Patch release adds multi-token prediction and GPU offloading while fixing Vulkan shader errors
New Features
- Multi-token prediction optimization for mixture-of-experts models
- GPU offloading for prompt processing on M5 and later Macs
Improvements
- Speed up generation at 16K context by 8% on Macs and 11% on NVIDIA GPUs
- Speed up prompt processing on M5+ Macs by running matrix multiplies on GPU
- Reduced working memory for Qwen3.5-4B from 3.7GB to 120MB
Bug Fixes
- Fix Vulkan GPU shader compilation errors for DFlash2 and Nemotron models
- Fix models failing to load on M1 and M2 Macs due to Metal kernel thread limits
- Fix memory pressure failures returning empty replies (now returns 503 with Retry-After)
Upgrade Notes
The 0.2.4 release drastically reduced memory usage for Qwen3.5-4B (3.7GB to 120MB), which may allow the model to run on lower-spec hardware. Ensure M5+ Macs are used to utilize the new GPU offloading feature for prompt processing.
The Two Releases
@magnitudedev/cli@0.2.4
Oct 2, 2026· 20 days ago
This patch release reduces memory overhead for large models and fixes startup and loading issues, particularly regarding kernel tuning on slower devices.
Full release notes@magnitudedev/cli@0.2.5
Oct 3, 2026· 19 days ago
This patch release focuses on performance optimizations for AI model generation and fixes compatibility issues on M1/M2 Macs and Vulkan GPUs.
Full release notes