v4.4.0

# 🎉 LocalAI 4.4.0 Release! 🚀

<h1 align="center">
  <br>
  <img height="300" src="https://raw.githubusercontent.com/mudler/LocalAI/refs/heads/master/core/http/static/logo.png">
  <br>
  <br>
</h1>

LocalAI 4.4.0 is out!

This is a big, **multimodal-and-distributed** release. Two brand-new audio backends land - **parakeet.cpp** (NVIDIA NeMo Parakeet ASR) and **CrispASR** (a multi-architecture ASR **and** TTS engine) - alongside native **object detection + segmentation** (`rfdetr-cpp`), **video understanding** in `llama-cpp`, and **LTX-2 video generation** in `stablediffusion-ggml`. Distributed mode grows up: **prefix-cache-aware routing** is on by default, and file transfers become **resumable**. There's a new **intelligent middleware** layer for request routing, PII filtering and cloud-model proxying, a **security hardening** pass that closes a credential-leak class across every outbound HTTP client, an interactive **`local-ai chat`** CLI, **RAG source citations** for agents, and a long run of reasoning / tool-call streaming fixes.

---

## 📌 TL;DR

| Area | Summary |
|------|---------|
| 🎙️ **Two new ASR backends** | `parakeet-cpp` (NeMo FastConformer TDT/CTC/RNNT, streaming, word/segment timestamps) and `crispasr` (many ASR architectures **+ TTS** in one binary). |
| 🧭 **Intelligent Middleware** | Capability-based model **routing**, **PII** detection/redaction, **cloud-model proxies** + a MITM proxy for subscription-auth Claude Code / Codex. |
| 🛰️ **Distributed v4** | Prefix-cache-aware routing (on by default), **NATS JWT auth + TLS/mTLS**, worker registration-token enforcement, resumable HTTP file transfers, boot-time model prefetch, ds4 layer-split inference. |
| 🎥 **Video, both ways** | Video **input** (understanding) in `llama-cpp` via mtmd, and video **generation** via **LTX-2** in `stablediffusion-ggml`. |
| 👁️ **Detection + Segmentation** | New native `rfdetr-cpp` backend (RF-DETR), 32 prebuilt GGUFs, bbox + per-detection PNG masks. |
| 🔐 **Outbound HTTP hardening** | `pkg/httpclient` refuses cross-host credential-leaking redirects across every outbound client (GHSA-3mj3-57v2-4636). |
| 🗣️ **TTS per-request control** | `instructions` + a generic `params` map plumbed end to end (Qwen3-TTS VoiceDesign / CustomVoice, Chatterbox). |
| 💻 **`local-ai chat`** | Interactive terminal chat against a running server, with `/models`, `/model`, `/clear`. |
| 📚 **RAG citations** | Agent answers now append a clickable `Sources:` block from the Knowledge Base. |
| 🧠 **Models** | Gemma 4 QAT family + QAT-matched **MTP** speculative-decoding bundles, Ideogram4, LTX-2.3 22B GGUFs. |

---

## 🚀 New Features & Major Enhancements

### 🎙️ Audio Gets Serious: Two New ASR Backends

This release doubles down on speech-to-text with two independent, cgo-less Go backends (purego, `CGO_ENABLED=0`), each shipping a full CI matrix, gallery importer and docs.

**`parakeet-cpp` - NVIDIA NeMo Parakeet (#10084).** Wraps [parakeet.cpp](https://github.com/mudler/parakeet.cpp), a C++/ggml port of NeMo Parakeet (FastConformer TDT/CTC/RNNT/hybrid) that matches the upstream PyTorch models on CPU. Text transcription, OpenAI-compatible **word timestamps**, and **cache-aware streaming** (16 kHz PCM chunks, `<EOU>`/`<EOB>` utterance boundaries). GGUFs for all 10 Parakeet models × 5 quants ship in [`mudler/parakeet-cpp-gguf`](https://huggingface.co/mudler/parakeet-cpp-gguf). Follow-ups in this cycle made it production-grade:

- **Dynamic batching (#10112)** - concurrent transcription requests are batched for throughput.
- **Real, NeMo-faithful segment timestamps (#10207)** - words are grouped into segments exactly like NeMo's `get_segment_offsets` (sentence-punctuation boundaries by default, opt-in `segment_gap_threshold` silence splitting in encoder frames). Streaming `FinalResult` segments now carry `start`/`end` when the library exposes the ABI v4 JSON entry points.
- **`nemotron-3.5-asr` multilingual streaming (#10199)** + per-request language selection.

**`crispasr` - many architectures + TTS in one backend (#10099).** Wraps [CrispASR](https://github.com/CrispStrobe/CrispASR) (a whisper.cpp/ggml fork, MIT) through its session C-ABI. One backend serves **ASR or TTS** depending on the loaded model, with the architecture auto-detected from the GGUF (or forced via `backend:`). The gallery gains **36 `-crispasr` entries (32 ASR + 4 TTS)**:

- **ASR** (e2e-verified across Whisper / Parakeet / Moonshine): parakeet, canary, cohere, qwen3, voxtral, granite, fastconformer-ctc, wav2vec2, hubert, data2vec, glm-asr, kyutai-stt, firered-asr, moonshine, mimo-asr, and more.
- **TTS** (all four e2e-verified to valid 24 kHz mono WAV): **vibevoice**, **chatterbox**, **qwen3-tts CustomVoice**, **orpheus** - via `backend:` / `codec:` / `speaker:` / `voice:` model options.


---

### 🧭 Intelligent Middleware: Routing, PII Filtering & Cloud Proxies

A new middleware layer (#9802) analyzes, routes, filters and transforms chat requests before they hit a model.

- **Capability-based routing.** Requests are classified (e.g. via an ArchRouter-style model) and scored across the capabilities they may require, then routed to the smallest model that satisfies them - easy requests go to small specialized models, hard or uncertain ones to larger general-purpose models. Classified embeddings are reused via cosine similarity so similar requests skip re-classification.
- **PII filtering.** Private information is detected per-pattern and can be **redacted, rerouted, or blocked**, with a streaming PII filter that preserves a buffered-emit invariant on `/v1/chat/completions`, Anthropic `/v1/messages`, and `/v1/completions`. A per-model PII pattern editor lives in the model config UI.
- **Cloud model proxies + MITM.** Cloud models and a MITM proxy can take part in routing/filtering - send easy requests to local models and hard ones to the cloud, and use **Claude Code / Codex subscriptions (OAuth)** through the PII filter via the MITM proxy (subject to provider ToS). Emits `proxy_connect` + `proxy_traffic` audit events and restores its listener from `runtime_settings.json` on restart.

Usage stats are recorded end to end and surfaced in REST, the UI, and MCP. Outbound clients used by this path were also the trigger for the security pass below.


---

### 🛰️ Distributed Mode v4

Distributed mode keeps maturing across routing, security and resilience.

**Prefix-cache-aware routing, on by default (#10071).** Routing now biases toward the replica that already holds the relevant KV/prefix cache, as a **load-guarded hint that never routes worse than today's round-robin**. A generic prefix tree (`pkg/radixtree`) maps cumulative prompt-prefix hashes to nodes; `core/services/nodes/prefixcache` turns the rendered prompt into a deterministic xxhash chain and makes a filter-then-score decision (narrow to load-eligible replicas, then prefer the longest-prefix match), feeding a `preferredNodeID` into the existing atomic `SELECT ... FOR UPDATE` pick. Observations sync across frontends over NATS. Round-robin is the floor; disable with `--distributed-prefix-cache=false`.

**NATS JWT auth + TLS/mTLS (#10159).** Previously anyone with access to the NATS port could publish backend-install messages or agent jobs (an SSRF / accidental-exposure risk). This adds JWT authentication and TLS/mTLS options, with workers acquiring and auto-refreshing their NATS credentials. Complemented by **worker file-transfer registration-token enforcement (#10183)**.

**Resumable file transfers (#10109).** Large model GGUFs over flaky/throttled links no longer restart from byte 0. The worker's `PUT /v1/files/<key>` honors `Content-Range` (308/416 resume semantics, `X-Content-SHA256` binding, final-hash verification) and the master-side stager HEAD-probes for the last accepted offset and resumes, switching to an outer time budget (`LOCALAI_FILE_TRANSFER_BUDGET`, default 1h) with exponential backoff.

**ds4 layer-split distributed inference (#10098).** Manual layer-split inference for the ds4 backend: a **coordinator** owns layers `0:K` and listens; **workers** dial in and own higher ranges, each loading only its slice of the GGUF (a new dependency-free `ds4-worker` binary, driven via `local-ai worker ds4-distributed`). Fully back-compatible when `ds4_role` is absent.

**Operational glue.** Boot-time gallery prefetch via `LOCALAI_PREFETCH_MODELS` (#10108); a gated `X-LocalAI-Node` response header for attribution (#9976); plus fixes: self-heal stale "model not loaded" routing (#10181), stage directory-based models to remote nodes (#10175), in-flight tracking for non-LLM methods - VAD, diarize, voice (#10238), reconciler survives frontend restarts (#9981), cross-replica OpCache sync (#9983), and the reinstall/upgrade UI no longer sticks on "reinstalling" (#10214).

---

### 🎥 Video, Both Directions

**Video input / understanding in `llama-cpp` (#10216).** Video-capable multimodal models (e.g. SmolVLM2-Video) can now be sent a video in a chat request, mirroring the existing image and audio paths. Tracks the upstream mtmd video landing (ggml-org/llama.cpp#24269); `grpc-server.cpp` forwards `request->videos()` into the mtmd `files` vector on both the template and non-template paths, and the React chat UI accepts `video/*`, renders an inline `<video controls>` player, and emits `video_url` content parts. `allow_video` is auto-gated by whether the loaded mmproj supports it. ffmpeg/ffprobe (already in the runtime image) extract frames.

**Video generation via LTX-2 (#9980).** `stablediffusion-ggml` wires `audio_vae_path` and `embeddings_connectors_path` through to the upstream LTX-2 fields, with a new `gallery/ltx-ggml.yaml` template (T2V / I2V / FLF2V recipes) and **six LTX-2.3 22B GGUF gallery entries** (dev + distilled, UD-Q4_K_M / Q4_K_M / Q8_0), each bundling the text encoder + video VAE + audio VAE + embeddings connectors. Follow-up fixes wired the `diffusion_model` flag and `vae_decode_only:false` for the i2v/flf2v paths (#9986, #9987) and muxed LTX-2 audio into the output MP4 (#9990).

---

### 👁️ Native Object Detection + Segmentation: `rfdetr-cpp`

A new Go native gRPC backend (#10028) dlopens `librfdetr.so` (built from [mudler/rf-detr.cpp](https://github.com/mudler/rf-detr.cpp)) and exposes the RF-DETR pipeline through LocalAI's `Detect` RPC. Supports all 5 detection variants (Nano…Large) and 3 segmentation variants (SegNano/SegSmall/SegMedium) at F32/F16/Q8_0/Q4_K, with **32 prebuilt GGUFs** on HuggingFace. Detection returns bbox + class_name + confidence; segmentation adds **per-detection PNG-encoded masks**. Matches PyTorch on CPU (sub-pixel bbox match, mask IoU 0.99+), with an HF gallery importer that auto-routes GGUF repos to the native backend.

> 🔗 PR: #10028. Also new: **Ideogram4** support in `stablediffusion-ggml` (#10201).


---

### 🗣️ TTS: Per-Request Instructions & Params

The OpenAI-compatible `/v1/audio/speech` `instructions` field was silently dropped at the HTTP→gRPC boundary, so style/voice could only come from static YAML. PR #10172 plumbs a generic per-request `instructions` string **and** an optional backend-specific `params` map end to end (proto, schema, `core/backend/tts.go`), unlocking per-line emotion/style (Qwen3-TTS CustomVoice, Chatterbox) and describe-a-voice (Qwen3-TTS VoiceDesign) from a single model config. Fully backward compatible - empty `instructions` falls back to YAML.

```bash
curl http://localhost:8080/v1/audio/speech -H "Content-Type: application/json" -d '{
  "model": "qwen-tts-design",
  "input": "Hello world, this is a test.",
  "instructions": "A calm, low-pitched elderly storyteller with a warm tone."
}'
```

Also: Qwen3-TTS request-language normalization for flexible matching (#10174), and LocalVQE **v1.3** with input/output spectrogram views in the Audio Transform UI (#10113).

---

### 🧠 Reasoning & Tool-Call Streaming Hardening

A focused run of correctness fixes for reasoning models and streaming tool calls:

- **`reasoning_effort` honored per request** and forwarded to the backend so jinja models can act on it (#10082, #10184).
- **`<think>` parsing**: stop `<think>` leaking into content in pure-content mode (#9991), stop a prefilled `<think>` from swallowing tag-less answers (#10225), and don't auto-enable self-spec MTP for draft-only assistant GGUFs (#10208).
- **Streaming + tools**: stop tool-call double-emission when the autoparser is active (#10055), stop tool-call JSON leaking into content on tokenizer-template models (#10057), validate auto-detected XML tool-call names with a robust glm-4.5/Hermes guard (#10059), and stop healing-marker stubs / prefill-misclassified content from corrupting the stream (#9999, #10000).

---

### 💻 `local-ai chat` + 📚 RAG Citations + 🛰️ Realtime

- **Interactive CLI chat (#10226).** A new opt-in `local-ai chat` command connects to a running server over the OpenAI-compatible API, streams completions, and supports `/models`, `/model <name>`, `/clear`, `/exit`. Keeps `local-ai run` focused on the server lifecycle. (Fixes #1535.)
- **RAG source citations (#10228).** When an agent answers from the Knowledge Base, the response now appends a clickable `Sources:` block listing the original documents - deduplicated per source, with the citation-free version saved to long-term memory. (Closes #9331.)
- **Configurable WebRTC ICE candidates (#10231).** New `LOCALAI_WEBRTC_NAT_1TO1_IPS` / `LOCALAI_WEBRTC_ICE_INTERFACES` knobs fix `/v1/realtime` calls dropping a few seconds in under Docker host networking (unroutable `docker0`/`veth` candidates).
- **"Fits in my GPU" filter (#10017)** on the Install Models page, plus a single shared `/api/operations` poller across UI consumers (#10029) and a React bundle code-split (#10042).

---

### 🧩 Backend Capability Registration & Startup Speed

- **Backend capability registration fixes** so the right backend is picked for the right job: register 5 backends missing from `BackendCapabilities` (#10107), and add face/speaker-recognition constants registering `insightface` + `speaker-recognition` (#10110).
- **Faster startup (#10213)**: skip vocab arrays and mmap GGUF headers during config parsing.

---

<details>
<summary> 
Click for the full changelog below!
</summary>


## What's Changed
### Bug fixes :bug:
* fix(config): register 5 backends missing from BackendCapabilities by @Dennisadira in https://github.com/mudler/LocalAI/pull/10107
* fix(config): register parakeet-cpp as a transcript backend (#9718) by @Dennisadira in https://github.com/mudler/LocalAI/pull/10106
* fix(parakeet-cpp): cublas/hipblas/vulkan builds were silently CPU-only by @localai-bot in https://github.com/mudler/LocalAI/pull/10120
* fix(nemo): pin texterrors to 1.1.6 for GLIBCXX compatibility by @fqscfqj in https://github.com/mudler/LocalAI/pull/10134
* fix(parakeet-cpp): convert audio before the non-batched transcribe path by @localai-bot in https://github.com/mudler/LocalAI/pull/10161
* fix(distributed): stage directory-based models to remote nodes by @localai-bot in https://github.com/mudler/LocalAI/pull/10175
* fix(config): add face/speaker recognition constants and register insightface + speaker-recognition by @Dennisadira in https://github.com/mudler/LocalAI/pull/10110
* fix(distributed): self-heal stale 'model not loaded' routing by @localai-bot in https://github.com/mudler/LocalAI/pull/10181
* fix(docs): use relearn notice shortcode instead of unsupported alert by @localai-bot in https://github.com/mudler/LocalAI/pull/10206
* fix(mtp): don't auto-enable self-spec MTP for draft-only assistant GGUFs by @localai-bot in https://github.com/mudler/LocalAI/pull/10208
* fix(config): skip vocab arrays and mmap GGUF headers to speed up startup by @Dennisadira in https://github.com/mudler/LocalAI/pull/10213
* fix: distributed backend reinstall/upgrade UI stuck on 'reinstalling' by @localai-bot in https://github.com/mudler/LocalAI/pull/10214
* fix(reasoning): stop prefilled <think> from swallowing tag-less answers by @localai-bot in https://github.com/mudler/LocalAI/pull/10225
* fix(cli): handle chat output errors by @Oceankj in https://github.com/mudler/LocalAI/pull/10229
* fix(distributed): track in-flight for non-LLM inference methods (VAD, diarize, voice, ...) by @localai-bot in https://github.com/mudler/LocalAI/pull/10238
### Exciting New Features 🎉
* feat: prefix-cache-aware routing for distributed mode by @localai-bot in https://github.com/mudler/LocalAI/pull/10071
* feat(ds4): layer-split distributed inference by @localai-bot in https://github.com/mudler/LocalAI/pull/10098
* feat(crispasr): add CrispASR backend — multi-architecture ASR + TTS by @localai-bot in https://github.com/mudler/LocalAI/pull/10099
* feat(worker): add LOCALAI_PREFETCH_MODELS for boot-time gallery prefetch by @localai-bot in https://github.com/mudler/LocalAI/pull/10108
* feat(distributed): resumable file uploads via HTTP Content-Range by @localai-bot in https://github.com/mudler/LocalAI/pull/10109
* feat(localvqe/audio): v1.3 release and add spectrograms to audio transform UI by @richiejp in https://github.com/mudler/LocalAI/pull/10113
* feat(parakeet-cpp): dynamic batching for concurrent transcription requests by @localai-bot in https://github.com/mudler/LocalAI/pull/10112
* feat(distributed): Add NATS JWT authentication and TLS/mTLS options by @richiejp in https://github.com/mudler/LocalAI/pull/10159
* feat(tts): support per-request instructions and params by @localai-bot in https://github.com/mudler/LocalAI/pull/10172
* feat(qwen3-tts-cpp): normalize request language for flexible matching by @localai-bot in https://github.com/mudler/LocalAI/pull/10174
* feat(distributed): enforce registration token for worker file transfer by @richiejp in https://github.com/mudler/LocalAI/pull/10183
* feat: forward reasoning_effort to the backend so jinja models honor it by @localai-bot in https://github.com/mudler/LocalAI/pull/10184
* Harden gallery-agent Hugging Face fetches against transient rate limiting by @Copilot in https://github.com/mudler/LocalAI/pull/10187
* feat(parakeet-cpp): nemotron-3.5-asr multilingual streaming model + request language support by @localai-bot in https://github.com/mudler/LocalAI/pull/10199
* feat: support Ideogram4 in stablediffusion-ggml backend + gallery by @localai-bot in https://github.com/mudler/LocalAI/pull/10201
* feat(parakeet-cpp): real segment timestamps (NeMo-faithful) by @localai-bot in https://github.com/mudler/LocalAI/pull/10207
* feat(llama-cpp): video input support (mtmd #24269) by @localai-bot in https://github.com/mudler/LocalAI/pull/10216
* feat(agents): surface KB source citations in RAG responses by @petechentw in https://github.com/mudler/LocalAI/pull/10228
* feat(cli): add interactive chat mode by @Oceankj in https://github.com/mudler/LocalAI/pull/10226
* feat(realtime): make WebRTC ICE candidates configurable by @localai-bot in https://github.com/mudler/LocalAI/pull/10231
### 🧠 Models
* chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/10163
* chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/10200
* chore(model gallery): :robot: add 1 new models via gallery agent by @localai-bot in https://github.com/mudler/LocalAI/pull/10209
* feat(gallery): add Gemma 4 QAT family + MTP speculative-decoding pairs by @localai-bot in https://github.com/mudler/LocalAI/pull/10215
### 📖 Documentation and examples
* docs: :arrow_up: update docs version mudler/LocalAI by @localai-bot in https://github.com/mudler/LocalAI/pull/10091
* docs: :arrow_up: update docs version mudler/LocalAI by @localai-bot in https://github.com/mudler/LocalAI/pull/10114
* docs: fix documentation typos by @Zhao73 in https://github.com/mudler/LocalAI/pull/10125
* docs(llama.cpp): note tensor split now works with quantized KV cache by @mudler in https://github.com/mudler/LocalAI/pull/10135
* docs: position LocalAI as a composable engine, not a bundle by @localai-bot in https://github.com/mudler/LocalAI/pull/10136
* docs: architecture & feature diagrams (blueprint style) by @localai-bot in https://github.com/mudler/LocalAI/pull/10137
* docs: fix distributed-mode diagram (workers use NATS, not PostgreSQL) by @localai-bot in https://github.com/mudler/LocalAI/pull/10138
### 👒 Dependencies
* chore: :arrow_up: Update ikawrakow/ik_llama.cpp to `3f40e73c367ad9f0c1b1819f28c7348c26aa340d` by @localai-bot in https://github.com/mudler/LocalAI/pull/10097
* chore: :arrow_up: Update antirez/ds4 to `ba00a8a88c4c5810a3d1fed6b7b8fa2b44b82fdc` by @localai-bot in https://github.com/mudler/LocalAI/pull/10095
* chore: :arrow_up: Update leejet/stable-diffusion.cpp to `d2797b86670622b6538123b4aeb5fbb6be2653c5` by @localai-bot in https://github.com/mudler/LocalAI/pull/10094
* chore: :arrow_up: Update ggml-org/llama.cpp to `d6588daa800058dfa54f1d7ea695b1a810c8ae18` by @localai-bot in https://github.com/mudler/LocalAI/pull/10093
* chore: :arrow_up: Update mudler/parakeet.cpp to `cb45f68068081af01e7092e91b038ee353eb56be` by @localai-bot in https://github.com/mudler/LocalAI/pull/10116
* chore: :arrow_up: Update ggml-org/whisper.cpp to `fe69461618ffc50ba8afa65c25cc6c6e34d4537f` by @localai-bot in https://github.com/mudler/LocalAI/pull/10117
* chore: :arrow_up: Update leejet/stable-diffusion.cpp to `be65ac7511b30379b003626c15224798929e33d4` by @localai-bot in https://github.com/mudler/LocalAI/pull/10118
* chore: :arrow_up: Update ggml-org/llama.cpp to `399739d5c5978351f39e3454bfbfbab4f369088f` by @localai-bot in https://github.com/mudler/LocalAI/pull/10119
* chore(model-gallery): :arrow_up: update checksum by @localai-bot in https://github.com/mudler/LocalAI/pull/10131
* chore: :arrow_up: Update ggml-org/whisper.cpp to `23ee03506a91ac3d3f0071b40e66a430eebdfa1d` by @localai-bot in https://github.com/mudler/LocalAI/pull/10130
* chore: :arrow_up: Update leejet/stable-diffusion.cpp to `7948df8ac1070f5f6881b8d34675821893eb97d6` by @localai-bot in https://github.com/mudler/LocalAI/pull/10127
* chore: :arrow_up: Update mudler/parakeet.cpp to `8a7c48209d7882a7ce79a6b306270e4703194543` by @localai-bot in https://github.com/mudler/LocalAI/pull/10129
* chore: :arrow_up: Update ggml-org/llama.cpp to `5dcb71166686799f0d873eab7386234302d05ecf` by @localai-bot in https://github.com/mudler/LocalAI/pull/10128
* chore: :arrow_up: Update CrispStrobe/CrispASR to `05e60432bcb5bc2113f8c395a41e86497c11504a` by @localai-bot in https://github.com/mudler/LocalAI/pull/10115
* chore(deps): bump github.com/mudler/edgevpn from 0.32.2 to 0.34.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/10153
* chore: :arrow_up: Update mudler/parakeet.cpp to `9edf17c3ada66e0f881dcff155492867db7ac4cf` by @localai-bot in https://github.com/mudler/LocalAI/pull/10141
* chore(deps): bump go.opentelemetry.io/otel/exporters/prometheus from 0.65.0 to 0.66.0 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/10151
* chore(deps): bump grpcio from 1.80.0 to 1.81.0 in /backend/python/vllm by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/10157
* chore(deps): bump github.com/google/go-containerregistry from 0.21.5 to 0.21.6 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/10149
* chore: :arrow_up: Update leejet/stable-diffusion.cpp to `2d40a8b2adcdf8b5b0ca0535f3bb7801b6ba13e5` by @localai-bot in https://github.com/mudler/LocalAI/pull/10144
* chore(deps): bump securego/gosec from 2.22.9 to 2.27.1 by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/10147
* chore: :arrow_up: Update ggml-org/whisper.cpp to `610e664ba7cfe3af46125ed1b5a1184fccb51bcd` by @localai-bot in https://github.com/mudler/LocalAI/pull/10140
* chore(deps): bump grpcio from 1.80.0 to 1.81.0 in /backend/python/transformers by @dependabot[bot] in https://github.com/mudler/LocalAI/pull/10158
* chore: :arrow_up: Update ggml-org/llama.cpp to `5c394fdc8b564eff6faacc50a139529d875f0e36` by @localai-bot in https://github.com/mudler/LocalAI/pull/10143
* chore: :arrow_up: Update antirez/ds4 to `477c0e82e2699b35a65fd0a1ed6fe66b41087dfe` by @localai-bot in https://github.com/mudler/LocalAI/pull/10142
* chore(model-gallery): :arrow_up: update checksum by @localai-bot in https://github.com/mudler/LocalAI/pull/10169
* chore: :arrow_up: Update ggml-org/llama.cpp to `94a220cd6745e6e3f8de62870b66fd5b9bc92700` by @localai-bot in https://github.com/mudler/LocalAI/pull/10168
* chore: :arrow_up: Update leejet/stable-diffusion.cpp to `1f9ee88e09c258053fa59d5e05e23dfb10fa0b13` by @localai-bot in https://github.com/mudler/LocalAI/pull/10166
* chore: :arrow_up: Update CrispStrobe/CrispASR to `13d54e110e1538e0f0bc3af0680b9ab246cfb48d` by @localai-bot in https://github.com/mudler/LocalAI/pull/10145
* chore: :arrow_up: Update predict-woo/qwen3-tts.cpp to `136e5d36c17083da0321fd96512dc7b263f94a44` by @localai-bot in https://github.com/mudler/LocalAI/pull/10165
* chore: :arrow_up: Update mudler/parakeet.cpp to `b11fe5bca78ad8b342dd559a43d76df3984bb447` by @localai-bot in https://github.com/mudler/LocalAI/pull/10167
* chore: :arrow_up: Update ikawrakow/ik_llama.cpp to `1520eda980564241434b791ce2bbbd128c4be9ea` by @localai-bot in https://github.com/mudler/LocalAI/pull/10180
* chore: :arrow_up: Update ggml-org/llama.cpp to `7c158fbb4aec1bdc9c81d6ca0e785139f4826fae` by @localai-bot in https://github.com/mudler/LocalAI/pull/10179
* chore: :arrow_up: Update ggml-org/whisper.cpp to `99613cb720b65036237d44b52f753b51f75c2797` by @localai-bot in https://github.com/mudler/LocalAI/pull/10178
* chore: :arrow_up: Update vllm-project/vllm cu130 wheel to `0.22.1` by @localai-bot in https://github.com/mudler/LocalAI/pull/10188
* chore: bump LocalAGI + localrecall (fix pgvector hybrid search seqscan, #10186) by @localai-bot in https://github.com/mudler/LocalAI/pull/10192
* chore: :arrow_up: Update mudler/parakeet.cpp to `843600590f96a31467a5199f827c253f34c110f7` by @localai-bot in https://github.com/mudler/LocalAI/pull/10198
* chore: :arrow_up: Update ikawrakow/ik_llama.cpp to `6b9de3dbaa21ae95ea80638e5ee836795cc48c93` by @localai-bot in https://github.com/mudler/LocalAI/pull/10190
* chore: :arrow_up: Update mudler/parakeet.cpp to `abd0087dcc92ec5ad1f96f9fd86c49eb26a5ce67` by @localai-bot in https://github.com/mudler/LocalAI/pull/10204
* chore: :arrow_up: Update ggml-org/whisper.cpp to `a8ec021f2750a473ff4a8f3883bc9fdf5feafa84` by @localai-bot in https://github.com/mudler/LocalAI/pull/10202
* chore(turboquant): bump to 7d9715f1 + fix compilation against rebased fork by @localai-bot in https://github.com/mudler/LocalAI/pull/10205
* chore: :arrow_up: Update ggml-org/llama.cpp to `31e82494c0a3913c919c1027fa70500fbf4c07dd` by @localai-bot in https://github.com/mudler/LocalAI/pull/10191
* chore: :arrow_up: Update mudler/parakeet.cpp to `e270af73b94c9a5c37ec516230219ed4580e1db6` by @localai-bot in https://github.com/mudler/LocalAI/pull/10212
* chore: :arrow_up: Update leejet/stable-diffusion.cpp to `b3d56d0ba1bd437886079e339118e8e75bb79ee7` by @localai-bot in https://github.com/mudler/LocalAI/pull/10211
* chore: :arrow_up: Update ggml-org/llama.cpp to `9e3b928fd8c9d14dbf15a8768b9fdd7e5c721d66` by @localai-bot in https://github.com/mudler/LocalAI/pull/10210
* chore: :arrow_up: Update antirez/ds4 to `c463029c205c2ec8d7ab6c0df4a3f52979091286` by @localai-bot in https://github.com/mudler/LocalAI/pull/10189
* chore: :arrow_up: Update CrispStrobe/CrispASR to `f7838a306687f22c281d29c250f879a4ab3df2d7` by @localai-bot in https://github.com/mudler/LocalAI/pull/10177
* chore: :arrow_up: Update antirez/ds4 to `512d07cb08f234b704b5a5959aa9e2d4c466eeb0` by @localai-bot in https://github.com/mudler/LocalAI/pull/10224
* chore: :arrow_up: Update ikawrakow/ik_llama.cpp to `2768b6251548b78b6610e95edad13f888ad95982` by @localai-bot in https://github.com/mudler/LocalAI/pull/10219
* chore: :arrow_up: Update leejet/stable-diffusion.cpp to `19bdfe22d255d5b4dff39d449318b9bc5ea2317f` by @localai-bot in https://github.com/mudler/LocalAI/pull/10222
* chore: :arrow_up: Update CrispStrobe/CrispASR to `97cad527d247edefc904e6c40c4cf5ee78bed055` by @localai-bot in https://github.com/mudler/LocalAI/pull/10221
* chore: :arrow_up: Update ggml-org/whisper.cpp to `df7638d8229a243af8a4b5a8ae557e0d74e0a0ae` by @localai-bot in https://github.com/mudler/LocalAI/pull/10220
* chore: :arrow_up: Update ikawrakow/ik_llama.cpp to `e6f8112f3ba126eed3ff5b30cdd08085414a7516` by @localai-bot in https://github.com/mudler/LocalAI/pull/10233
* chore: :arrow_up: Update antirez/ds4 to `91bafb5acd5a6cf00b1e55ef68bf40ddd207bee7` by @localai-bot in https://github.com/mudler/LocalAI/pull/10234
* chore: :arrow_up: Update ggml-org/llama.cpp to `039e20a2db9e87b2477c76cc04905f3e1acad77f` by @localai-bot in https://github.com/mudler/LocalAI/pull/10223
* chore: :arrow_up: Update CrispStrobe/CrispASR to `c29f6653a516a3001d923944dad8892072cc7334` by @localai-bot in https://github.com/mudler/LocalAI/pull/10236
### Other Changes
* refactor(routing): extract replica picker into pkg/clusterrouting by @localai-bot in https://github.com/mudler/LocalAI/pull/10123
* test(react-ui): add page render-smoke specs, reset the coverage gate by @richiejp in https://github.com/mudler/LocalAI/pull/10122

</details>

---

## 🙌 New Contributors

* @TLoE419 made their first contribution in https://github.com/mudler/LocalAI/pull/9978
* @fqscfqj made their first contribution in https://github.com/mudler/LocalAI/pull/10012
* @bozhouDev made their first contribution in https://github.com/mudler/LocalAI/pull/10055
* @Oceankj made their first contribution in https://github.com/mudler/LocalAI/pull/10019
* @Zhao73 made their first contribution in https://github.com/mudler/LocalAI/pull/10125
* @petechentw made their first contribution in https://github.com/mudler/LocalAI/pull/10228

Enjoy!

---

**Full Changelog**: https://github.com/mudler/LocalAI/compare/v4.3.0...v4.4.0