v1.6.0
looplj/axonhubv1.6.0May 27, 2026by amirivojdan
AI Summary
This release introduces transliteration support between Farsi and Tajik scripts, new text transforms for data augmentation and loanword replacement, an Iran mirror for faster model downloads, and a refactored conjugation engine, alongside enhancements to the Shekar Studio Web UI.
Key Highlights
- Farsi ↔ Tajik Transliteration using a quantized ByT5 model
- Persianizer transform for replacing loanwords with native Persian alternatives
- Text Noise Transforms (Keyboard, OCR, Whitespace) for data augmentation
- Rule-Based Informal Classifier for detecting colloquial Persian text
- Iran Mirror with latency-based selection for faster model downloads
Breaking Changes
- Dropped Python 3.10 support; package now requires Python 3.11+.
New Features
- FarsiToTajik and TajikToFarsi transliteration functions
- Persianizer transform with 1,700+ loanword mappings
- NumberToWords transform for numeric to word conversion
- KeyboardNoise, OCRNoise, and WhitespaceNoise transforms
- RuleBasedInformalClassifier for colloquial text detection
- Automated mirror selection (Shekar AI) for improved download speeds
- Transliteration panel in Shekar Studio Web UI
- Localization updates including English and Persian translations
Full Release Notes
# What's New in v1.6.0 This release introduces transliteration support, a suite of new text transforms for data augmentation and Persianization, faster and more reliable model downloads, and a refactored conjugation system. It also includes several enhancements to the Shekar Studio Web UI. ## Highlights - **Farsi ↔ Tajik Transliteration** – added a new `transliteration` module powered by a quantized ByT5 model. Use `FarsiToTajik` and `TajikToFarsi` (both exposed at the package top level) to convert between Persian (Arabic script) and Tajik (Cyrillic script). - **Persianizer for Loanword Replacement** – the new `Persianizer` transform suggests and substitutes native Persian alternatives for foreign loanwords, backed by a curated mapping of 1,700+ entries sourced from beparsi.com. Use `Persianizer()` to auto-replace, or `.suggest()` to get ranked alternatives with positions. - **NumberToWords Transform** – the new `NumberToWords` transform converts numeric digits (both Persian and Arabic-Indic) into their Persian word form (e.g. `۱۲۳` → `صد و بیست و سه`). - **Text Noise Transforms for Data Augmentation** – three new transforms for synthetic noise generation, useful for training robust models and benchmarking: * `KeyboardNoise` – simulates typos based on Persian keyboard adjacency (substitution, insertion, deletion, repeat, shift). * `OCRNoise` – simulates OCR errors based on visual character confusions. * `WhitespaceNoise` – corrupts whitespace and ZWNJ structure (deletion or swapping between space ↔ ZWNJ). All three accept independent per-operation probabilities and a `seed` for reproducibility. - **Rule-Based Informal Classifier** – added `RuleBasedInformalClassifier` for fast, dependency-free detection of colloquial/informal Persian text using keyword matching over informal vocabulary and conjugated verb forms. Inspired by [Persian-Informal-Text-Detector](https://github.com/MahtaFetrat/Persian-Informal-Text-Detector). - **Iran Mirror with Latency-Based Selection** – the model hub now selects the fastest available mirror automatically (currently `shekar.ai` and `ir.shekar.ai`), significantly improving download speed for users in Iran and reducing failures when one mirror is unreachable. - **Shekar Studio Enhancements** – the built-in web interface (`shekar serve`) now includes: * A new **Transliteration** panel with Persian (Tajik) support * Localized UI with **English** and **Persian (Tajik)** translations alongside Persian (Farsi) - **Refactored Conjugation Engine** – conjugation logic has been split into separate formal and informal pipelines for cleaner code, better test coverage, and more accurate informal verb generation. ## Other Changes - Dropped Python 3.10 support; the package now requires **Python 3.11+**. - Bumped `onnxruntime` minimum to `>=1.26.0`. - Changed `YaNormalizer` default to 'standard'.