Starstruk Logo MarkStarstruk

2026.4.1

poteto/hiring-without-whiteboards2026.4.1Apr 24, 2026by GabrielBianconi

AI Summary

Gateway update introducing async observability defaults, TypeScript evaluators, and enhanced OpenTelemetry trace exports.

Key Highlights

  • Breaking change: Gateway defaults to async observability writes.
  • Deprecation: Removed TensorZero Autopilot 'Sessions' page.
  • Added aggregated variant usage data (tokens, cost) to the UI.
  • Added support for vLLM's new reasoning field.

Breaking Changes

  • The gateway now defaults to async observability writes to reduce tail latency.

New Features

  • Added TypeScript evaluators for inference evaluations.
  • Added support for vLLM's new `reasoning` field.
  • Added aggregated variant usage data (tokens, cost, etc.) to the UI.
  • Added inference cost data to exported OpenTelemetry traces.
  • Added `export.otlp.traces.include_content` configuration field.

Full Release Notes

> [!CAUTION]
> **Breaking Changes**
>
> - The gateway now defaults to async observability writes to reduce tail latency: inferences are sent to the client before they are persisted in the database. To restore the previous behavior, set `observability.async_writes = false`. **[[docs]](https://www.tensorzero.com/docs/gateway/configuration-reference)**

> [!WARNING]
> **Deprecations**
>
> - Removed the TensorZero Autopilot "Sessions" page from the UI. We recently added a TensorZero MCP that integrates nicely with coding agents, and we'll re-introduce advanced TensorZero Autopilot workflows in a platform-agnostic format soon.

**Bug Fixes**

- Return HTTP code 429 for rate limiting errors.
- Fixed a bug affecting ClickHouse database names with hyphens. (thanks @ianliuy!)

**New Features**

- Added TypeScript evaluators (for inference evaluations).
- Added support for vLLM's new `reasoning` field.
- Added aggregated variant usage data (tokens, cost, etc.) to the UI.
- Added inference cost data to exported OpenTelemetry traces. (thanks @kimsehwan96!)
- Added `export.otlp.traces.include_content` (default false) configuration field to include inference content (e.g. prompts, messages) in exported OpenTelemetry GenAI traces.

_& multiple under-the-hood and UI improvements_