v0.4.0
feyninc/chonkiev0.4.0Oct 10, 2026by github-actions[bot]
Upgrading?
AI Summary
Selective porting of upstream Laya v0.4.1 runtime features to native MLX inference, introducing parallel option layout support, new confidence gates, and fixes for language detection and memory management.
Key Highlights
- Native MLX inference support for upstream features
- Router behavior changes: language-undecided inputs now go to multilingual checkpoint by default
- Support for parallel option layouts throughout MLX inference and compilation
- Added predict_tournament function for narrowing large choice sets
- Fixes for FP16 kernel crashes on macOS 26 using FP32 intermediate computation
Breaking Changes
- Router() now sends language-undecided inputs to the multilingual checkpoint by default; set Router(default="english") to retain the previous English fallback behavior
New Features
- predict_tournament function
- Scalar and per-option-count min_confidence gates for Agent and Router
- Support for parallel-layout checkpoints in MLX inference and caching
- Custom checkpoint registration methods (register, unregister, registered, models=)
- Release of MLX free-buffer caches on unload
Full Release Notes
Laya-MLX v0.4.0 selectively ports upstream Laya v0.4.1 runtime features to native MLX inference. ## Behavior change `Router()` now sends language-undecided inputs to the multilingual checkpoint. Identified English still uses the English checkpoint. Set `Router(default="english")` to retain the previous fallback behavior. ## Changes - Keep resident-model requests and status reads available during cold checkpoint construction. Deduplicate builds and synchronize targeted unloads per checkpoint. - Register custom checkpoints with `register`, `unregister`, `registered`, or `models=`. Source replacement and unregistration cannot resurrect an outdated in-flight model. - Add `predict_tournament` to narrow large choice sets without an embedding model. Its probabilities and usage describe the final pass; metadata records finalists and elimination rounds. - Add scalar and per-option-count `min_confidence` gates to Agent and Router. Gate results are reported without replacing the original answer or automatically invoking a fallback. - Use FP32 intermediate computation for CPU parallel-option attention to avoid native FP16 kernel crashes on hosted macOS 26; GPU precision and parameter storage precision are unchanged. - Support checkpoints trained with `option_layout="parallel"` throughout MLX inference, compilation, prefix caching, and conversion. Existing checkpoints remain sequential; do not enable parallel layout by editing an old checkpoint's configuration. - Fix language detection for capitalized acronyms, address fragments, emphasis capitals, and mixed scripts; fix French device-footer removal. - Release MLX free-buffer caches on unload and eviction while preserving caller-owned and in-flight model references. - Update English/Chinese documentation, attribution, and CI with separate pinned sequential and parallel-layout upstream references. ## Validation - 231 tests passed on CPU and 231 on GPU before release. - Three published checkpoints in FP32 and FP16: 378/378 argmax agreements with the pinned sequential upstream reference. - 100 deterministic finite calls per configuration, 600 total, with zero measured active-memory growth. - Parallel-layout tests cover all 24 permutations of four options in FP32/FP16, numerical comparison with upstream v0.4.1, compilation, caching, and export/reload. - Ruff, wheel/sdist strict metadata checks, and installed-wheel smoke tests passed. [Raw checkpoint validation](https://github.com/mizorewww/laya-mlx/blob/v0.4.0/benchmarks/results/upstream-v041-validation.json) · [API and migration details](https://github.com/mizorewww/laya-mlx/blob/v0.4.0/README.md#upstream-v041-runtime-port) Install or upgrade: ```bash python -m pip install --upgrade laya-mlx==0.4.0 ```
Related Repositories
public-apis/public-apis★ 491,213
A collective list of free APIs
NousResearch/hermes-agent★ 256,078
The agent that grows with you
yt-dlp/yt-dlp★ 200,250
A feature-rich command-line audio/video downloader
microsoft/markitdown★ 191,556
Python tool for converting files and office documents to Markdown.
Significant-Gravitas/AutoGPT★ 187,699
AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.