v0.4.0
SaschaHeyer/gen-ai-livestreamv0.4.0Oct 10, 2026by github-actions[bot]
Upgrading?
AI Summary
Ports upstream Laya v0.4.1 runtime features including parallel option layout support and FP32 intermediate computation to avoid native kernel crashes.
Key Highlights
- **Behavior change**: `Router()` now sends language-undecided inputs to the multilingual checkpoint
- Adds `predict_tournament` to narrow large choice sets without an embedding model
- Adds `min_confidence` gates to Agent and Router
- Supports `option_layout="parallel"` for checkpoints
- Uses FP32 intermediate computation for CPU parallel-option attention
Breaking Changes
- Router() behavior change: language-undecided inputs now default to multilingual checkpoint
New Features
- `predict_tournament` function
- `min_confidence` gates
- Parallel option layout support
- FP32 intermediate computation
Full Release Notes
Laya-MLX v0.4.0 selectively ports upstream Laya v0.4.1 runtime features to native MLX inference. ## Behavior change `Router()` now sends language-undecided inputs to the multilingual checkpoint. Identified English still uses the English checkpoint. Set `Router(default="english")` to retain the previous fallback behavior. ## Changes - Keep resident-model requests and status reads available during cold checkpoint construction. Deduplicate builds and synchronize targeted unloads per checkpoint. - Register custom checkpoints with `register`, `unregister`, `registered`, or `models=`. Source replacement and unregistration cannot resurrect an outdated in-flight model. - Add `predict_tournament` to narrow large choice sets without an embedding model. Its probabilities and usage describe the final pass; metadata records finalists and elimination rounds. - Add scalar and per-option-count `min_confidence` gates to Agent and Router. Gate results are reported without replacing the original answer or automatically invoking a fallback. - Use FP32 intermediate computation for CPU parallel-option attention to avoid native FP16 kernel crashes on hosted macOS 26; GPU precision and parameter storage precision are unchanged. - Support checkpoints trained with `option_layout="parallel"` throughout MLX inference, compilation, prefix caching, and conversion. Existing checkpoints remain sequential; do not enable parallel layout by editing an old checkpoint's configuration. - Fix language detection for capitalized acronyms, address fragments, emphasis capitals, and mixed scripts; fix French device-footer removal. - Release MLX free-buffer caches on unload and eviction while preserving caller-owned and in-flight model references. - Update English/Chinese documentation, attribution, and CI with separate pinned sequential and parallel-layout upstream references. ## Validation - 231 tests passed on CPU and 231 on GPU before release. - Three published checkpoints in FP32 and FP16: 378/378 argmax agreements with the pinned sequential upstream reference. - 100 deterministic finite calls per configuration, 600 total, with zero measured active-memory growth. - Parallel-layout tests cover all 24 permutations of four options in FP32/FP16, numerical comparison with upstream v0.4.1, compilation, caching, and export/reload. - Ruff, wheel/sdist strict metadata checks, and installed-wheel smoke tests passed. [Raw checkpoint validation](https://github.com/mizorewww/laya-mlx/blob/v0.4.0/benchmarks/results/upstream-v041-validation.json) · [API and migration details](https://github.com/mizorewww/laya-mlx/blob/v0.4.0/README.md#upstream-v041-runtime-port) Install or upgrade: ```bash python -m pip install --upgrade laya-mlx==0.4.0 ```
Related Repositories
rasbt/LLMs-from-scratch★ 107,095
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
lyogavin/airllm★ 40,274
AirLLM 2.8T LLM inference with single 4GB GPU
NirDiamant/RAG_Techniques★ 29,967
This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. Each technique has a detailed notebook tutorial.
microsoft/OmniParser★ 25,540
A simple screen parsing tool towards pure vision based GUI agent
karpathy/nn-zero-to-hero★ 24,911
Neural Networks: Zero to Hero