livekit-agents@1.6.5

clusterzx/paperless-ailivekit-agents@1.6.5Jul 9, 2026by tinalenguyen

AI Summary

A major update introducing expressive mode and connection pooling, alongside extensive bug fixes for STT/TTS providers and voice pipeline stability.

Key Highlights

  • Added expressive mode for voice agents with <expr/> markers.
  • Implemented connection pooling for xAI TTS to improve performance.
  • Broadened support for Gemini 3 Flash model variants.
  • Fixed critical voice pipeline issues including tool call race conditions and in-flight audio handling.
  • Added Protoface avatar plugin and Rime speed_alpha support.

Breaking Changes

  • Removed structured LLM output from the voice pipeline.

New Features

  • Expressive mode for voice agents
  • xAI TTS connection pooling
  • Protoface avatar plugin
  • Rime speed_alpha support
  • SmallestAI word_timestamps for TTS

Full Release Notes

## What's Changed
* fix(google): broaden _is_gemini_3_flash_model to cover all Gemini 3 Flash variants by @balazssandor in https://github.com/livekit/agents/pull/6210
* fix(assemblyai): respect `mode` preset for turn-silence; remap deprecated `u3-pro` by @dlange-aai in https://github.com/livekit/agents/pull/6220
* Move frame processor url/token/stream info to client sdk by @1egoman in https://github.com/livekit/agents/pull/5867
* fix(worker): exit when connection fails after max retries by @longcw in https://github.com/livekit/agents/pull/6103
* fix(voice): don't re-issue in-flight tool calls on a new turn by @longcw in https://github.com/livekit/agents/pull/6227
* (phonic): Use stream_ahead_of_real_time mode for Phonic WebSocket and sample rate 24000 by @qionghuang6 in https://github.com/livekit/agents/pull/6236
* fix(anthropic): use non-whitespace trailing dummy to satisfy provider validation by @JasonGordonD in https://github.com/livekit/agents/pull/6216
* fix(deepgram): sync missing STT/TTS API params, fix duplicate languages, add TTS model list by @Namit1867 in https://github.com/livekit/agents/pull/5362
* chore(deps): update dependency msgpack to v1.2.1 [security] by @renovate[bot] in https://github.com/livekit/agents/pull/6243
* docs: fix hotel_receptionist architecture section after refactor by @detail-app[bot] in https://github.com/livekit/agents/pull/6230
* fix(deepgram): remove duplicate redact param by @longcw in https://github.com/livekit/agents/pull/6245
* fix(openai): delete trimmed empty audio items by @he-yufeng in https://github.com/livekit/agents/pull/5887
* fix(openai realtime): discard orphaned response after interrupt or timeout by @longcw in https://github.com/livekit/agents/pull/6244
* fix CI due to realtime_model test by @davidzhao in https://github.com/livekit/agents/pull/6261
* fix: preserve transcription item id by @nightcityblade in https://github.com/livekit/agents/pull/6120
* fix(openai): skip realtime truncate when no audio was played fixes : #6157 by @ByteMaster-1 in https://github.com/livekit/agents/pull/6158
* feat(assemblyai): add language_code streaming param for language steering by @gsharp-aai in https://github.com/livekit/agents/pull/6219
* fix(xai): emit interim transcripts for in-progress status in realtime by @ByteMaster-1 in https://github.com/livekit/agents/pull/6272
* conversation-aware STT recognition (keyterms + chat context) by @longcw in https://github.com/livekit/agents/pull/6039
* Allow LemonSlice avatars to join third party meeting platforms by @aweitz in https://github.com/livekit/agents/pull/6233
* (xai realtime): pass voice parameter by @tinalenguyen in https://github.com/livekit/agents/pull/6276
* feat(certifi): user certifi as a fallback when system store is not present by @chenghao-mou in https://github.com/livekit/agents/pull/6231
* chore(eot): update log message about slow stt transcript by @chenghao-mou in https://github.com/livekit/agents/pull/6179
* fix(fallback_adapter): update stream start time offset in stt fallbackadapter by @chenghao-mou in https://github.com/livekit/agents/pull/6246
* feat(voice): stream tool call status events by @longcw in https://github.com/livekit/agents/pull/6100
* set default shutdown reason by @davidzhao in https://github.com/livekit/agents/pull/6284
* bump worker drain timeout to 1hr by @chenghao-mou in https://github.com/livekit/agents/pull/6286
* feat(rime): support speed_alpha for arcana and coda models by @balazssandor in https://github.com/livekit/agents/pull/6287
* feat(deepgram): apply Flux option updates in-band by @longcw in https://github.com/livekit/agents/pull/6277
* feat: realtime model fallback adapter by @longcw in https://github.com/livekit/agents/pull/6282
* update keyterm detection default model by @longcw in https://github.com/livekit/agents/pull/6289
* fix(eot): tighten eot cancellation by speech acitivity by @chenghao-mou in https://github.com/livekit/agents/pull/6274
* silero: use onnx when explicitely using silero.VAD.load() by @theomonnom in https://github.com/livekit/agents/pull/6290
* fix(openai): emit start/end of speech events for realtime STT AGT-3051 by @chenghao-mou in https://github.com/livekit/agents/pull/6297
* feat(smallestai): word_timestamps for TTS, v4 STT endpoints, eou_timeout fix by @harshitajain165 in https://github.com/livekit/agents/pull/5953
* fix(aws): nest endpointingSensitivity under turnDetectionConfiguration for Nova Sonic 2 by @St-Luciferr in https://github.com/livekit/agents/pull/6201
* switch to blingfire tokenizer by default by @davidzhao in https://github.com/livekit/agents/pull/6301
* feat(plugins-tavus): rename replica/persona to face/pal by @carolin-tavus in https://github.com/livekit/agents/pull/6239
* fix(openai): avoid mutating raw tool schemas for Responses by @VectorPeak in https://github.com/livekit/agents/pull/6305
* feat(openai): add temperature parameter to STT by @longcw in https://github.com/livekit/agents/pull/6307
* fix(liveavatar): preserve provider status errors by @VectorPeak in https://github.com/livekit/agents/pull/6306
* fix incorrect attribution of latency in tts.ttfb by @davidzhao in https://github.com/livekit/agents/pull/6300
* update livekit to 1.1.13 by @davidzhao in https://github.com/livekit/agents/pull/6311
* fix(voice): race between session.run and an in-flight AgentTask handoff by @longcw in https://github.com/livekit/agents/pull/6315
* fix(voice): resolve video task leak in AgentSession by @Mayank-MSJ-Singh in https://github.com/livekit/agents/pull/6318
* Add `remove()` method to ChatContext for flexible item removal by @3eid in https://github.com/livekit/agents/pull/5131
* fix: expose speech generation errors via SpeechHandle.exception() by @longcw in https://github.com/livekit/agents/pull/6304
* chore(deps): update actions/checkout action to v7 by @renovate[bot] in https://github.com/livekit/agents/pull/6325
* fix(test): stop false-positive "Audio data isn't PCM" failures in TTS tests by @u9g in https://github.com/livekit/agents/pull/6331
* feat(anam) add director notes config by @sr-anam in https://github.com/livekit/agents/pull/6267
* fix(aws): preserve image format for Bedrock images by @VectorPeak in https://github.com/livekit/agents/pull/6328
* (feat) 11Labs STT : Add enable_logging by @Ludobaka in https://github.com/livekit/agents/pull/6273
* feat: expressive mode by @tinalenguyen in https://github.com/livekit/agents/pull/6116
* Add connection pooling for xAI TTS by @godququ5-code in https://github.com/livekit/agents/pull/6317
* voice: output retries for run(output_type=...) by @theomonnom in https://github.com/livekit/agents/pull/6080
* disable openai sdk max_retries by @chenghao-mou in https://github.com/livekit/agents/pull/6341
* Allow direct image uploads for LemonSlice avatars by @jp-lemon in https://github.com/livekit/agents/pull/6343
* remove structured LLM output from voice pipeline by @tinalenguyen in https://github.com/livekit/agents/pull/6345
* remove livekit-blingfire (moved) by @theomonnom in https://github.com/livekit/agents/pull/6320
* (expressive mode): <expr/> expression marker dialect by @tinalenguyen in https://github.com/livekit/agents/pull/6347
* (expressive mode): fix markup tags impacting pacing by @tinalenguyen in https://github.com/livekit/agents/pull/6350
* feat(core): add plain_text_content property for ChatMessage by @chenghao-mou in https://github.com/livekit/agents/pull/6348
* fix(tts): restart ChunkedStream retries under a fresh request_id by @u9g in https://github.com/livekit/agents/pull/6346
* send ServerInfo to CLI host on start by @davidzhao in https://github.com/livekit/agents/pull/6260
* docs: fix livekit-durable README title copied from removed blingfire plugin by @detail-app[bot] in https://github.com/livekit/agents/pull/6355
* Feat/updating documentation for Gnani  by @Gnani-AI-Mintlify in https://github.com/livekit/agents/pull/6004
* feat(soniox): support speed parameter in TTS by @yui0303 in https://github.com/livekit/agents/pull/6339
* fix(azure): recover from STT error cancellations instead of hanging by @apoorva-01 in https://github.com/livekit/agents/pull/6362
* fix: honor IGNORE_ON_ENTER for tools inside a Toolset by @longcw in https://github.com/livekit/agents/pull/6361
* fix: send DTMF to the active session room by @longcw in https://github.com/livekit/agents/pull/6360
* docs(soniox): add speed parameter to TTS example in README by @detail-app[bot] in https://github.com/livekit/agents/pull/6366
* (expressive mode): hide and remove from examples by @tinalenguyen in https://github.com/livekit/agents/pull/6370
* fix(core): flip text content fields AGT-3103 by @chenghao-mou in https://github.com/livekit/agents/pull/6353
* livekit-agents@1.6.5 by @github-actions[bot] in https://github.com/livekit/agents/pull/6371

## New Contributors
* @balazssandor made their first contribution in https://github.com/livekit/agents/pull/6210
* @1egoman made their first contribution in https://github.com/livekit/agents/pull/5867
* @JasonGordonD made their first contribution in https://github.com/livekit/agents/pull/6216
* @ByteMaster-1 made their first contribution in https://github.com/livekit/agents/pull/6158
* @aweitz made their first contribution in https://github.com/livekit/agents/pull/6233
* @St-Luciferr made their first contribution in https://github.com/livekit/agents/pull/6201
* @carolin-tavus made their first contribution in https://github.com/livekit/agents/pull/6239
* @VectorPeak made their first contribution in https://github.com/livekit/agents/pull/6305
* @Mayank-MSJ-Singh made their first contribution in https://github.com/livekit/agents/pull/6318
* @3eid made their first contribution in https://github.com/livekit/agents/pull/5131
* @godququ5-code made their first contribution in https://github.com/livekit/agents/pull/6317
* @yui0303 made their first contribution in https://github.com/livekit/agents/pull/6339
* @apoorva-01 made their first contribution in https://github.com/livekit/agents/pull/6362

**Full Changelog**: https://github.com/livekit/agents/compare/livekit-agents@1.6.4...livekit-agents@1.6.5