Starstruk Logo MarkStarstruk

v2.4.0

pydantic/pydantic-aiv2.4.0Jul 3, 2026by dsfaccini

AI Summary

This release introduces significant enhancements to evaluation infrastructure, including `GEval` and span-based evaluators, while improving security by splitting file handling options and fixing various bugs across different providers.

Key Highlights

  • Added `GEval` evaluator and standard quality metric rubrics for `LLMJudge`.
  • Added agentic span-based evaluators (ToolCorrectness, TrajectoryMatch, etc.).
  • Split `preserve_file_data` into `allow_uploaded_files` (inbound security) and an AG-UI representation opt-in.
  • Fixed `XSearchTool` handle limit and other provider-specific bugs.
  • Cleaned up pending tasks and streams on `KeyboardInterrupt`.

New Features

  • `GEval` evaluator
  • Span-based evaluators (ToolCorrectness, TrajectoryMatch, etc.)
  • `allow_uploaded_files` security feature

Full Release Notes

<!-- Release notes generated using configuration in .github/release.yml at main -->

## What's Changed
### 🚀 Features
* Split `preserve_file_data` into `allow_uploaded_files` (inbound security) and an AG-UI representation opt-in by @DouweM in https://github.com/pydantic/pydantic-ai/pull/6232
* Add `GEval` evaluator and standard quality metric rubrics for `LLMJudge` by @dmontagu in https://github.com/pydantic/pydantic-ai/pull/5129
* Add agentic span-based evaluators (ToolCorrectness, TrajectoryMatch, ArgumentCorrectness, MaxToolCalls, MaxModelRequests) by @dmontagu in https://github.com/pydantic/pydantic-ai/pull/5130
### 🐛 Bug Fixes
* fix(xai): update `XSearchTool` handle limit by @gyx09212214-prog in https://github.com/pydantic/pydantic-ai/pull/5979
* Handle None metadata in Gradio weather agent example by @hramezani in https://github.com/pydantic/pydantic-ai/pull/6214
* fix(mistral): preserve extended usage fields from API response by @felipebridge in https://github.com/pydantic/pydantic-ai/pull/5693
* Avoid mutable default argument in `process_tool_calls` by @kratos0718 in https://github.com/pydantic/pydantic-ai/pull/6189
* feat: forward `WebSearchTool.user_location` to xAI web search by @manan-tech in https://github.com/pydantic/pydantic-ai/pull/6181
* Clean up the pending task and streams on `KeyboardInterrupt` in synchronous `run_until_complete` wrappers by @DouweM in https://github.com/pydantic/pydantic-ai/pull/6198
* fix(tool-search): drop redundant `description` from `ToolSearchMatch` by @serozhenka in https://github.com/pydantic/pydantic-ai/pull/5654
* fix(temporal): pass through genai_prices and httpx2 for workflow cost() by @syf2211 in https://github.com/pydantic/pydantic-ai/pull/6219
* fix(evals): reject non-positive `max_concurrency` by @VectorPeak in https://github.com/pydantic/pydantic-ai/pull/6228
* Lazy-start `run_stream_events()` background task on first event iteration by @DouweM in https://github.com/pydantic/pydantic-ai/pull/6234
### 📦 Dependencies
* Bump `genai-prices` to >=0.0.69 by @adtyavrdhn in https://github.com/pydantic/pydantic-ai/pull/6212

## New Contributors
* @raullenchai made their first contribution in https://github.com/pydantic/pydantic-ai/pull/5008
* @gyx09212214-prog made their first contribution in https://github.com/pydantic/pydantic-ai/pull/5979
* @felipebridge made their first contribution in https://github.com/pydantic/pydantic-ai/pull/5693
* @kratos0718 made their first contribution in https://github.com/pydantic/pydantic-ai/pull/6189
* @serozhenka made their first contribution in https://github.com/pydantic/pydantic-ai/pull/5654

**Full Changelog**: https://github.com/pydantic/pydantic-ai/compare/v2.3.0...v2.4.0