Starstruk Logo MarkStarstruk

2026.4.1

LaurenzV/hayro2026.4.1Apr 24, 2026by GabrielBianconi

AI Summary

This release introduces a breaking change to default async observability writes and deprecates the Autopilot Sessions UI, while adding new evaluation and tracing capabilities.

Key Highlights

  • The gateway now defaults to async observability writes to reduce tail latency.
  • Added TypeScript evaluators for inference evaluations.
  • Added support for vLLM's new `reasoning` field.
  • Added aggregated variant usage data (tokens, cost) to the UI.
  • Added configuration field to include inference content in OTLP traces.

Breaking Changes

  • The gateway now defaults to async observability writes: inferences are sent to the client before they are persisted in the database. To restore the previous behavior, set `observability.async_writes = false`.

New Features

  • TypeScript evaluators
  • Support for vLLM's new `reasoning` field
  • Aggregated variant usage data
  • Inference cost data to exported OpenTelemetry traces
  • Configuration field `export.otlp.traces.include_content`

Full Release Notes

> [!CAUTION]
> **Breaking Changes**
>
> - The gateway now defaults to async observability writes to reduce tail latency: inferences are sent to the client before they are persisted in the database. To restore the previous behavior, set `observability.async_writes = false`. **[[docs]](https://www.tensorzero.com/docs/gateway/configuration-reference)**

> [!WARNING]
> **Deprecations**
>
> - Removed the TensorZero Autopilot "Sessions" page from the UI. We recently added a TensorZero MCP that integrates nicely with coding agents, and we'll re-introduce advanced TensorZero Autopilot workflows in a platform-agnostic format soon.

**Bug Fixes**

- Return HTTP code 429 for rate limiting errors.
- Fixed a bug affecting ClickHouse database names with hyphens. (thanks @ianliuy!)

**New Features**

- Added TypeScript evaluators (for inference evaluations).
- Added support for vLLM's new `reasoning` field.
- Added aggregated variant usage data (tokens, cost, etc.) to the UI.
- Added inference cost data to exported OpenTelemetry traces. (thanks @kimsehwan96!)
- Added `export.otlp.traces.include_content` (default false) configuration field to include inference content (e.g. prompts, messages) in exported OpenTelemetry GenAI traces.

_& multiple under-the-hood and UI improvements_