v0.4.0
mizorewww/laya-mlxv0.4.0Oct 10, 2026by github-actions[bot]
Upgrading?
AI Summary
This release selectively ports upstream Laya v0.4.1 runtime features to native MLX inference, introducing support for parallel-layout checkpoints and CPU parallel-option attention while fixing language detection and integration issues.
Key Highlights
- Introduction of `predict_tournament` for narrowing choice sets without embedding models.
- Support for parallel-layout checkpoints throughout MLX inference, compilation, and caching.
- FP32 intermediate computation for CPU parallel-option attention to fix macOS 26 crashes.
- New `min_confidence` gates for Agents and Routers (scalar and per-option-count).
- Improved language detection handling for acronyms, fragments, and mixed scripts.
Breaking Changes
- `Router()` behavior changed: language-undecided inputs now go to the multilingual checkpoint by default. Users must set `Router(default="english")` to restore the previous fallback behavior.
New Features
- `predict_tournament` function to narrow large choice sets.
- Scalar and per-option-count `min_confidence` gates for Agent and Router.
- Full support for `option_layout="parallel"` checkpoints in MLX inference, compilation, prefix caching, and conversion.
Full Release Notes
Laya-MLX v0.4.0 selectively ports upstream Laya v0.4.1 runtime features to native MLX inference. ## Behavior change `Router()` now sends language-undecided inputs to the multilingual checkpoint. Identified English still uses the English checkpoint. Set `Router(default="english")` to retain the previous fallback behavior. ## Changes - Keep resident-model requests and status reads available during cold checkpoint construction. Deduplicate builds and synchronize targeted unloads per checkpoint. - Register custom checkpoints with `register`, `unregister`, `registered`, or `models=`. Source replacement and unregistration cannot resurrect an outdated in-flight model. - Add `predict_tournament` to narrow large choice sets without an embedding model. Its probabilities and usage describe the final pass; metadata records finalists and elimination rounds. - Add scalar and per-option-count `min_confidence` gates to Agent and Router. Gate results are reported without replacing the original answer or automatically invoking a fallback. - Use FP32 intermediate computation for CPU parallel-option attention to avoid native FP16 kernel crashes on hosted macOS 26; GPU precision and parameter storage precision are unchanged. - Support checkpoints trained with `option_layout="parallel"` throughout MLX inference, compilation, prefix caching, and conversion. Existing checkpoints remain sequential; do not enable parallel layout by editing an old checkpoint's configuration. - Fix language detection for capitalized acronyms, address fragments, emphasis capitals, and mixed scripts; fix French device-footer removal. - Release MLX free-buffer caches on unload and eviction while preserving caller-owned and in-flight model references. - Update English/Chinese documentation, attribution, and CI with separate pinned sequential and parallel-layout upstream references. ## Validation - 231 tests passed on CPU and 231 on GPU before release. - Three published checkpoints in FP32 and FP16: 378/378 argmax agreements with the pinned sequential upstream reference. - 100 deterministic finite calls per configuration, 600 total, with zero measured active-memory growth. - Parallel-layout tests cover all 24 permutations of four options in FP32/FP16, numerical comparison with upstream v0.4.1, compilation, caching, and export/reload. - Ruff, wheel/sdist strict metadata checks, and installed-wheel smoke tests passed. [Raw checkpoint validation](https://github.com/mizorewww/laya-mlx/blob/v0.4.0/benchmarks/results/upstream-v041-validation.json) · [API and migration details](https://github.com/mizorewww/laya-mlx/blob/v0.4.0/README.md#upstream-v041-runtime-port) Install or upgrade: ```bash python -m pip install --upgrade laya-mlx==0.4.0 ```
Related Repositories
public-apis/public-apis★ 491,135
A collective list of free APIs
NousResearch/hermes-agent★ 255,963
The agent that grows with you
yt-dlp/yt-dlp★ 200,054
A feature-rich command-line audio/video downloader
microsoft/markitdown★ 191,491
Python tool for converting files and office documents to Markdown.
Significant-Gravitas/AutoGPT★ 187,703
AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.