v0.9.5
SaschaHeyer/gen-ai-livestreamv0.9.5Sep 3, 2026by tashfeenahmed
Upgrading?
AI Summary
Introduces task-type-aware routing for code and chat, adds a quota forecast endpoint, and enables persistent SQLite caching. Security features include a new encryption key rotation script.
Key Highlights
- Task-type-aware routing using `x-freellm-task-type` header.
- New `/v1/quota-forecast` endpoint for daily quota headroom.
- Response cache can persist to SQLite (opt-out with environment variable).
- New encryption key rotation script (`npm run rotate-encryption-key`).
- MCP endpoint has an on/off switch.
New Features
- Task-type-aware routing
- Quota forecasting
- Persistent SQLite cache
- Encryption key rotation
- MCP toggle
Full Release Notes
## Routing - Task-type-aware routing: send `x-freellm-task-type: code` or `chat` to nudge the bandit weights toward intelligence or speed for that request. Code is also detected from unambiguous markers in the prompt. The `fastest`, `reliable` and `custom` strategies are never adjusted. (#1128) - The share of the weight shift is tunable via `GET/PUT /api/settings/task-weight-share`; `0` turns it off. (#1129) ## Models and quota - `/v1/models` now reports `execution_status` per model (`ready`, `needsKey`, `exhausted`) using the router's own cooldown, rate-window and key-scope checks, and accepts `?execution_status=` as a filter. (#1101) - New `GET /v1/quota-forecast` endpoint: remaining daily headroom per quota pool with a low-balance flag before the UTC reset. (#1111) ## Cache - The response cache can now persist to SQLite so it survives restarts. Opt out with `RESPONSE_CACHE_PERSIST=0`. Only active when the cache itself is enabled, and note that it stores model responses on disk in plaintext. (#1097) - Analytics shows a cache card with hit rate and saved tokens when the cache is on. (#1117) ## MCP - The `/mcp` endpoint has an on/off switch (`GET/PUT /api/settings/enable-mcp`) and a toggle under Keys → agent compatibility. Existing installs stay on; fresh installs start with it off. A disabled server answers 403 on every verb. (#1073) ## Security and operations - `npm run rotate-encryption-key` re-encrypts every stored secret (provider keys, proxy credentials, client profile keys, Fetch Relay token) under a new `ENCRYPTION_KEY`, with `--dry-run`. (#1113) - CI: GitHub Actions bumped to current majors and pinned to full commit SHAs, Dependabot keeps them fresh, server test coverage is reported on the Node 22 leg, and ESLint now runs for the server and CLI workspaces. (#1067, #1118, #1119, #1121) ## Fixes - `setup-opencode` writes `limit.output`, which OpenCode's config schema requires. Shipped as CLI 0.5.1 on npm. (#1055) ## Docs - QwenPaw setup guide in English and Chinese. Thanks @qinghuanandejiangshi. (#1137) Thanks to @suantea and @yfdyh000 for the contributions in this release. Full changelog: https://github.com/tashfeenahmed/freellmapi/compare/v0.9.4...v0.9.5 --- ⭐ Like the free router? [Go Premium](https://freellmapi.co/?utm_source=github&utm_medium=release&utm_campaign=premium&utm_content=v0.9.5#pricing) — the live signed catalog, $19/yr, cancel anytime.
Related Repositories
rasbt/LLMs-from-scratch★ 107,079
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
lyogavin/airllm★ 40,215
AirLLM 2.8T LLM inference with single 4GB GPU
NirDiamant/RAG_Techniques★ 29,962
This repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. Each technique has a detailed notebook tutorial.
microsoft/OmniParser★ 25,539
A simple screen parsing tool towards pure vision based GUI agent
karpathy/nn-zero-to-hero★ 24,907
Neural Networks: Zero to Hero