cli@1.4.0
the-dev-tools/dev-toolscli@1.4.0Oct 10, 2026by github-actions[bot]
Upgrading?
AI Summary
This release integrates AI checks into the engine and enhances streaming capabilities, enabling automated quality validation and real-time event processing for flow runs.
Key Highlights
- AI checks (schema, latency, G-Eval) are now integrated into the engine and run automatically.
- Server-Sent Events (SSE) and streaming responses are now fully supported.
- New response properties allow access to events, TTFT, and usage data.
- Checks are cached and configurable via environment variables.
New Features
- Automated AI quality checks using `expect:` blocks.
- Streaming support for OpenAI, Anthropic, Vercel AI, and SSE providers.
- New response properties: `events`, `ttft_ms`, `event_count`.
- Configurable `stream_timeout_ms` for long-running streams.
Full Release Notes
## 1.4.0 (2026-10-10)
### 🚀 Features
- AI checks are now part of the engine. A request or graphql step's `expect:` block (the Stresseur syntax, unchanged) is validated on import and evaluated by `devtools flow run` after the flow's steps finish, so it never counts in step timings. Results are printed under the flow's table and written to the JSON report as each step's `checks[]` (`kind`, `passed`, `score`, `reason`, `judge_model`, `cached`, `value`, `limit`). ([4b494b2f](https://github.com/the-dev-tools/dev-tools/commit/4b494b2f))
- **Checks:** `schema`/`schema_at`, `max_latency_ms`, the new `max_ttft_ms`, token budgets from `usage`, and a G-Eval `judge` (Anthropic or any OpenAI-compatible API). The judge uses injection-resistant prompting, logprob weighting or borderline re-sampling, and runs in parallel.
- **Judge key:** `STRESSEUR_JUDGE_API_KEY`, `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`, from the environment or `./.env`. Without one, judge checks are skipped with a notice.
- **Cache:** verdicts are cached in `--judge-cache`, default `.devtools/judge-cache.json`.
- **Exit code:** failed checks fail a flow only with `quality: { fail_below: … }`.
Server-Sent Events responses are now read as they arrive. A text/event-stream response streams automatically; a request step's `stream: openai | anthropic | vercel-ai | sse` picks the preset that assembles the text, and `stream_timeout_ms` bounds an open stream (default 30 s). Assertions, `expect:` and later steps can read `response.text`, `response.events`, `response.event_count`, `response.ttft_ms` and the streamed `response.usage`.
### ❤️ Thank You
- moosebayRelated Repositories
ollama/ollama★ 184,219
Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
fatedier/frp★ 110,057
A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.
infiniflow/ragflow★ 92,621
RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context layer for LLMs
gohugoio/hugo★ 90,199
The world’s fastest framework for building websites.
junegunn/fzf★ 83,715
:cherry_blossom: A command-line fuzzy finder