cli@1.4.0

the-dev-tools/dev-toolscli@1.4.0Oct 10, 2026by github-actions[bot]

Upgrading?

AI Summary

This release integrates AI checks into the engine and enhances streaming capabilities, enabling automated quality validation and real-time event processing for flow runs.

Key Highlights

  • AI checks (schema, latency, G-Eval) are now integrated into the engine and run automatically.
  • Server-Sent Events (SSE) and streaming responses are now fully supported.
  • New response properties allow access to events, TTFT, and usage data.
  • Checks are cached and configurable via environment variables.

New Features

  • Automated AI quality checks using `expect:` blocks.
  • Streaming support for OpenAI, Anthropic, Vercel AI, and SSE providers.
  • New response properties: `events`, `ttft_ms`, `event_count`.
  • Configurable `stream_timeout_ms` for long-running streams.

Full Release Notes

## 1.4.0 (2026-10-10)

### 🚀 Features

- AI checks are now part of the engine. A request or graphql step's `expect:` block (the Stresseur syntax, unchanged) is validated on import and evaluated by `devtools flow run` after the flow's steps finish, so it never counts in step timings. Results are printed under the flow's table and written to the JSON report as each step's `checks[]` (`kind`, `passed`, `score`, `reason`, `judge_model`, `cached`, `value`, `limit`). ([4b494b2f](https://github.com/the-dev-tools/dev-tools/commit/4b494b2f))

  - **Checks:** `schema`/`schema_at`, `max_latency_ms`, the new `max_ttft_ms`, token budgets from `usage`, and a G-Eval `judge` (Anthropic or any OpenAI-compatible API). The judge uses injection-resistant prompting, logprob weighting or borderline re-sampling, and runs in parallel.
  - **Judge key:** `STRESSEUR_JUDGE_API_KEY`, `ANTHROPIC_API_KEY` or `OPENAI_API_KEY`, from the environment or `./.env`. Without one, judge checks are skipped with a notice.
  - **Cache:** verdicts are cached in `--judge-cache`, default `.devtools/judge-cache.json`.
  - **Exit code:** failed checks fail a flow only with `quality: { fail_below: … }`.

  Server-Sent Events responses are now read as they arrive. A text/event-stream response streams automatically; a request step's `stream: openai | anthropic | vercel-ai | sse` picks the preset that assembles the text, and `stream_timeout_ms` bounds an open stream (default 30 s). Assertions, `expect:` and later steps can read `response.text`, `response.events`, `response.event_count`, `response.ttft_ms` and the streamed `response.usage`.

### ❤️ Thank You

- moosebay

Related Repositories