livekit-agents@1.5.0

## Highlights

### Preemptive generation is now enabled by default
Preemptive generation starts LLM and TTS inference before the end of a user’s turn is detected, reducing overall latency.

To disable it:

```python
session = AgentSession(preemptive_generation=False)
```

### Adaptive Interruption Handling

The headline feature of v1.5.0: an audio-based ML model that distinguishes genuine user interruptions from incidental sounds like backchannels ("mm-hmm"), coughs, sighs, or background noise. Enabled by default — no configuration needed.

Key stats:
- **86% precision** and **100% recall** at 500ms overlapping speech
- Rejects **51%** of traditional VAD false positives
- Detects true interruptions **64% faster** than VAD alone
- Inference completes in **30ms or less**

When a false interruption is detected, the agent automatically resumes playback from where it left off — no re-generation needed.

To opt out and use VAD-only interruption:

```python
session = AgentSession(
    ...
    turn_handling=TurnHandlingOptions(
        interruption={
            "mode": "vad",
        },
    ),
)
```

Blog post: https://livekit.com/blog/adaptive-interruption-handling

### Dynamic Endpointing

Endpointing delays now adapt to each conversation's natural rhythm. Instead of a fixed silence threshold, the agent uses an exponential moving average of pause durations to dynamically adjust when it considers the user's turn complete.

```python
session = AgentSession(
    ...
    turn_handling=TurnHandlingOptions(
        endpointing={
            "mode": "dynamic",
            "min_delay": 0.3,
            "max_delay": 3.0,
        },
    ),
)
```

### New `TurnHandlingOptions` API

Endpointing and interruption settings are now consolidated into a single `TurnHandlingOptions` dict passed to `AgentSession`. Old keyword arguments (`min_endpointing_delay`, `allow_interruptions`, etc.) still work but are deprecated and will emit warnings.

```python
session = AgentSession(
    turn_handling={
        "turn_detection": "vad",
        "endpointing": {"min_delay": 0.5, "max_delay": 3.0},
        "interruption": {"enabled": True, "mode": "adaptive"},
    },
)
```

### Session Usage Tracking

New `SessionUsageUpdatedEvent` provides structured, per-model usage data — token counts, character counts, and audio durations — broken down by provider and model:

```python
@session.on("session_usage_updated")
def on_usage(ev: SessionUsageUpdatedEvent):
    for usage in ev.usage.model_usage:
        print(f"{usage.provider}/{usage.model}: {usage}")
```

Usage types: `LLMModelUsage`, `TTSModelUsage`, `STTModelUsage`, `InterruptionModelUsage`.

You can also access aggregated usage at any time via the `session.usage` property:

```python
usage = session.usage
for model_usage in usage.model_usage:
    print(model_usage)
```

Usage data is also included in `SessionReport` (via `model_usage`), so it's available in post-session telemetry and reporting out of the box.

### Per-Turn Latency on `ChatMessage.metrics`

Each `ChatMessage` now carries a `metrics` field (`MetricsReport`) with per-turn latency data:
- `transcription_delay` — time to obtain transcript after end of speech
- `end_of_turn_delay` — time between end of speech and turn decision
- `on_user_turn_completed_delay` — time in the developer callback

### Action-Aware Chat Context Summarization

Context summarization now includes function calls and their outputs when building summaries, preserving tool-use context across the conversation window.

### Configurable Log Level

Set the agent log level via `LIVEKIT_LOG_LEVEL` environment variable or through `ServerOptions`, without touching your code.

## Deprecations

| Deprecated | Replacement | Notes |
|---|---|---|
| `metrics_collected` event | `session_usage_updated` event + `ChatMessage.metrics` | Usage/cost data moves to `session_usage_updated`; per-turn latency moves to `ChatMessage.metrics`. Old listeners still work with a deprecation warning. |
| `UsageCollector` | `ModelUsageCollector` | New collector supports per-model/provider breakdown |
| `UsageSummary` | `LLMModelUsage`, `TTSModelUsage`, `STTModelUsage` | Typed per-service usage classes |
| `RealtimeModelBeta` | `RealtimeModel` | Beta API removed |
| `AgentFalseInterruptionEvent.message` / `.extra_instructions` | Automatic resume via adaptive interruption | Accessing these fields logs a deprecation warning |
| `AgentSession` kwargs: `min_endpointing_delay`, `max_endpointing_delay`, `allow_interruptions`, `discard_audio_if_uninterruptible`, `min_interruption_duration`, `min_interruption_words`, `turn_detection`, `false_interruption_timeout`, `resume_false_interruption` | `turn_handling=TurnHandlingOptions(...)` | Old kwargs still work but emit deprecation warnings. Will be removed in v2.0. |
| `Agent` / `AgentTask` kwargs: `turn_detection`, `min_endpointing_delay`, `max_endpointing_delay`, `allow_interruptions` | `turn_handling=TurnHandlingOptions(...)` | Same migration path as `AgentSession`. Will be removed in future versions. |

## Complete changelog
* (xai): add grok text to speech api to readme by @tinalenguyen in https://github.com/livekit/agents/pull/5125
* Remove Gemini 2.0 models from inference gateway types by @Shubhrakanti in https://github.com/livekit/agents/pull/5133
* feat: support log level via ServerOptions and LIVEKIT_LOG_LEVEL env var by @onurburak9 in https://github.com/livekit/agents/pull/5112
* fix: preserve 'type' field in TaskGroup JSON schema enum items by @weiguangli-io in https://github.com/livekit/agents/pull/5073
* feat(assemblyai): expose session ID from Begin event by @dlange-aai in https://github.com/livekit/agents/pull/5132
* fix: strip empty {} entries from anyOf/oneOf in strict JSON schema by @theomonnom in https://github.com/livekit/agents/pull/5137
* fix: update_instructions() now reflected in tool call response generation by @weiguangli-io in https://github.com/livekit/agents/pull/5072
* Make chat context summarization action-aware by @toubatbrian in https://github.com/livekit/agents/pull/5099
* fix(realtime): sync remote items to local chat_ctx with placeholders to prevent in-flight deletion by @longcw in https://github.com/livekit/agents/pull/5114
* Set _speech_start_time when VAD START_OF_SPEECH activates by @hudson-worden in https://github.com/livekit/agents/pull/5027
* Fix(inworld): "Context not found" errors caused by invalid enum parameter types by @ianbbqzy in https://github.com/livekit/agents/pull/5153
* increase generate_reply timeout & remove RealtimeModelBeta by @theomonnom in https://github.com/livekit/agents/pull/5149
* add livekit-blockguard plugin by @theomonnom in https://github.com/livekit/agents/pull/5023
* openai: add max_completion_tokens to with_azure() by @abhishekranjan-bluemachines in https://github.com/livekit/agents/pull/5143
* Restrict mistralai dependency to use v1 sdk by @csanz91 in https://github.com/livekit/agents/pull/5116
* feat(assemblyai): add DEBUG-level diagnostic logging by @dlange-aai in https://github.com/livekit/agents/pull/5146
* Fix Phonic `generate_reply` to resolve with the current `GenerationCreatedEvent` by @qionghuang6 in https://github.com/livekit/agents/pull/5147
* fix(11labs): add empty keepalive message and remove final duplicates by @chenghao-mou in https://github.com/livekit/agents/pull/5139
* AGT-2182: Add adaptive interruption handling and dynamic endpointing by @chenghao-mou in https://github.com/livekit/agents/pull/4771
* livekit-agents 1.5.0 by @theomonnom in https://github.com/livekit/agents/pull/5165

## New Contributors
* @onurburak9 made their first contribution in https://github.com/livekit/agents/pull/5112
* @weiguangli-io made their first contribution in https://github.com/livekit/agents/pull/5073
* @abhishekranjan-bluemachines made their first contribution in https://github.com/livekit/agents/pull/5143
* @csanz91 made their first contribution in https://github.com/livekit/agents/pull/5116

**Full Changelog**: https://github.com/livekit/agents/compare/livekit-agents@1.4.6...livekit-agents@1.5.0