v1.0.0
khoj-ai/khojv1.0.0May 10, 2026by KoljaB
AI Summary
A major version 1.0 release introducing a new Transcription Engine System, new ASR engines, a FastAPI streaming server, and syllable-boundary scheduling.
Key Highlights
- New Transcription Engine System with lazy-loading and multiple backend support
- New ASR Engines including Whisper CPP, OpenAI Whisper, Moonshine, and Parakeet
- FastAPI Browser Streaming Server with multi-user support
- Syllable-Boundary Realtime Scheduling to reduce ASR calls
Breaking Changes
- Updated legacy browser client WebSocket port from 8001 to 9001
- AudioToTextRecorder usage now requires explicit extras installation (e.g., `[faster-whisper]`)
New Features
- New Transcription Engine System
- New ASR Engines (Whisper CPP, OpenAI Whisper, Moonshine, etc.)
- FastAPI Browser Streaming Server
- Syllable-Boundary Realtime Scheduling
- RealtimeTextStabilizer for text stability
Full Release Notes
# RealtimeSTT v1.0.0 ### Major Features - **New Transcription Engine System** - Added a lazy-loaded engine abstraction under `RealtimeSTT/transcription_engines`. - `faster_whisper` remains the compatibility default. - Final and realtime transcription can now use different backends via `transcription_engine` and `realtime_transcription_engine`. - Backend-specific settings can be passed separately with `transcription_engine_options` and `realtime_transcription_engine_options`. - **New ASR Engines** - Added `whisper_cpp` through `pywhispercpp`. - Added `openai_whisper` for OpenAI’s local Whisper package. - Added Moonshine, NVIDIA Parakeet/NeMo, Granite Speech, Qwen3-ASR, and Cohere Transcribe adapters. - Added CPU INT8 sherpa-onnx paths for Parakeet and Moonshine. - Added optional Kroko-ONNX aliases: `kroko_onnx`, `kroko`, and `banafo_kroko`. - **FastAPI Browser Streaming Server** - Added `example_fastapi_server`, a browser-based streaming STT server with binary WebSocket audio input. - Added multi-user support with session-local audio buffers, VAD state, transcript state, segment IDs, clear/reset, warnings, errors, realtime text, and final text. - Heavy ASR models are shared through final/realtime inference lanes instead of loading one model per browser session. - Added fair per-session scheduling, realtime coalescing, stale realtime job dropping, final-job preservation, capacity limits, `/health`, `/api/config`, and `/api/metrics`. - Added wake-word support in the FastAPI server, including Porcupine/OpenWakeWord options and a follow-up window after wake activation. - **Syllable-Boundary Realtime Scheduling** - Added optional CPU-only acoustic boundary scheduling with `realtime_transcription_use_syllable_boundaries`. - The scheduler triggers realtime ASR near likely syllable/vowel boundaries, with configurable sensitivity and follow-up delays. - In the reference fixture, realtime ASR calls dropped from 239 to 44, an 81.6% reduction, while final WER stayed at 0.000. ### Improvements - **Customizable Installation** - `pip install RealtimeSTT` now installs the core recorder, audio, WebRTC VAD, and Silero VAD dependencies. - Optional ASR and wake-word stacks are exposed as extras, including `faster-whisper`, `whisper-cpp`, `openai-whisper`, `sherpa-onnx`, `parakeet`, `qwen`, `porcupine`, `openwakeword`, `wakewords`, `recommended`, and `all`. - Wake-word dependencies are no longer mandatory unless selected. WebRTC VAD and Silero VAD remain core dependencies because the recorder still initializes both VAD paths. - **Realtime Text Stability** - Added a structured `RealtimeTextStabilizer` with stable deltas, unstable preview text, outlier handling, finalization checks, and diagnostics. - Added `on_realtime_text_stabilization_update` for applications that need structured realtime stabilization events. - **Docs And Examples** - Reworked the README into a shorter entry point and moved detailed guidance into `docs/`. - Added docs for installation, configuration, transcription engines, wake words, external audio, FastAPI server usage, troubleshooting, testing, and architecture. - Added per-engine documentation for faster-whisper, whisper.cpp, OpenAI Whisper, Moonshine, sherpa-onnx, Parakeet/NeMo, Transformers engines, Cohere, and Kroko-ONNX. ### Fixes - **Recorder Stability: Slow Final Transcription** - Fixed a recorder bug where slow CPU final transcription could cause following speech to be missed while `text()` was blocked. - Completed recordings are now queued, continuous listening stays armed during final transcription, delayed Silero confirmation is tolerated, and finite `feed_audio()` streams can be flushed. - The CPU/int8 regression now preserves all 9/9 utterances with combined WER 0.000. - **Recorder Stability: Stale Pre-Roll Audio** - Fixed stale pre-recording audio leaking from one utterance into the next final transcription. - The pre-roll buffer is now cleared when a recording transitions to stopped, while normal fresh pre-roll behavior is preserved. - **Server Isolation And Backpressure** - Transcript events are routed only to their owning browser session. - Session slots are reserved before recorder construction to prevent connection bursts from over-instantiating recorders. - Added per-session backlog limits and long-audio force-finalization to keep the server responsive under load. ### Other - Updated Docker/CUDA packaging to CUDA 12.8.1/cuDNN on Ubuntu 24.04 with PyTorch/Torchaudio 2.7.1 cu128. - Split GPU Torch requirements into `requirements-gpu-torch.txt`. - Updated the legacy browser client WebSocket port from 8001 to 9001. - Added ignored/excluded local development folders, model caches, test results, private docs/tests, backups, and secrets to `.gitignore`/`MANIFEST.in`. - Added GitHub funding metadata. - Package version bumped to `1.0.0`. ### Compatibility Notes - Existing `AudioToTextRecorder` usage with the default faster-whisper backend remains the intended compatibility path, but environments should now install the needed extras explicitly, for example `pip install "RealtimeSTT[faster-whisper]"`. - Some new engines are experimental or platform-sensitive. Kroko-ONNX is registered and contract-tested, but real Windows validation was blocked by upstream native build issues; use Linux, WSL2, or Docker for runtime validation.