v0.0.82
pipecat-ai/pipecatv0.0.82Aug 28, 2025by aconchillo
AI Summary
This release introduces a new `LLMRunFrame` to trigger LLM responses and a 'universal' `LLMContext` system to enable runtime LLM switching. It also adds features for skipping TTS and configuring pipeline task timeouts.
Key Highlights
- Introduced `LLMRunFrame` to trigger LLM responses and `LLMSwitcher` for runtime switching
- Added universal `LLMContext` and `LLMContextAggregatorPair` to share context between LLMs
- Added `skip_tts` field to `TextFrame` and `LLMConfigureOutputFrame` for text-only modes
- Added `cancel_timeout_secs` to `PipelineTask`
Breaking Changes
- `OpenAILLMContextFrame` usage replaced by `LLMRunFrame` in common patterns
- `CartesiaTTSService` speed parameter changed from float to enum (`slow`, `normal`, `fast`)
New Features
- Universal `LLMContext` (OpenAI/Google supported initially)
- `LLMSwitcher` class
- `LLMService.run_inference()` method
- `skip_tts` functionality
- `cancel_timeout_secs` parameter
Full Release Notes
### Added
- Added a new `LLMRunFrame` to trigger an LLM response:
```python
await task.queue_frames([LLMRunFrame()])
```
This replaces `OpenAILLMContextFrame`, which you’d previously typically use like this:
```python
await task.queue_frames([context_aggregator.user().get_context_frame()])
```
Use this way of kicking off your conversation when you’ve already initialized your context and are simply instructing the bot when to go:
```python
context = OpenAILLMContext(messages, tools)
context_aggregator = llm.create_context_aggregator(context)
# ...
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
# Kick off the conversation.
await task.queue_frames([LLMRunFrame()])
```
Note that if you want to add new messages when kicking off the conversation, you could use `LLMMessagesAppendFrame` with `run_llm=True` instead:
```python
@transport.event_handler("on_client_connected")
async def on_client_connected(transport, client):
# Kick off the conversation.
await task.queue_frames([LLMMessagesAppendFrame(new_messages, run_llm=True)])
```
In the rare case you don’t have a context aggregator in your pipeline, then you may continue using a context frame.
- Added support for switching between audio+text to text-only modes within the same pipeline. This is done by pushing `LLMConfigureOutputFrame(skip_tts=True)` to enter text-only mode, and disabling it to return to audio+text. The LLM will still generate tokens and add them to the context, but they will not be sent to TTS.
- Added `skip_tts` field to `TextFrame`. This lets a text frame bypass TTS while still being included in the LLM context. Useful for cases like structured text that isn’t meant to be spoken but should still contribute to context.
- Added a `cancel_timeout_secs` argument to `PipelineTask` which defines how long the pipeline has to complete cancellation. When `PipelineTask.cancel()` is called, a `CancelFrame` is pushed through the pipeline and must reach the end. If it does not reach the end within the specified time, a warning is shown and the wait is aborted.
- Added a new "universal" (LLM-agnostic) `LLMContext` and accompanying `LLMContextAggregatorPair`, which will eventually replace `OpenAILLMContext` (and the other under-the-hood contexts) and the other context aggregators. The new universal `LLMContext` machinery allows a single context to be shared between different LLMs, enabling runtime LLM switching and scenarios like failover.
From the developer's point of view, switching to using the new universal context machinery will usually be a matter of going from this:
```python
context = OpenAILLMContext(messages, tools)
context_aggregator = llm.create_context_aggregator(context)
```
To this:
```python
context = LLMContext(messages, tools)
context_aggregator = LLMContextAggregatorPair(context)
```
To start, the universal `LLMContext` is supported with the following LLM services:
- `OpenAILLMService`
- `GoogleLLMService`
- Added a new `LLMSwitcher` class to enable runtime LLM switching, built atop a new generic `ServiceSwitcher`.
Switchers take a switching strategy. The first available strategy is `ServiceSwitcherStrategyManual`.
To switch LLMs at runtime, the LLMs must be sharing one instance of the new universal `LLMContext` (see above bullet).
```python
# Instantiate your LLM services
llm_openai = OpenAILLMService(api_key=os.getenv("OPENAI_API_KEY"))
llm_google = GoogleLLMService(api_key=os.getenv("GOOGLE_API_KEY"))
# Instantiate a switcher
# (ServiceSwitcherStrategyManual defaults to OpenAI, as it's first in the list)
llm_switcher = LLMSwitcher(
llms=[llm_openai, llm_google], strategy_type=ServiceSwitcherStrategyManual
)
# Create your pipeline
pipeline = Pipeline(
[
transport.input(),
stt,
context_aggregator.user(),
llm_switcher,
tts,
transport.output(),
context_aggregator.assistant(),
]
)
task = PipelineTask(pipeline, params=PipelineParams(allow_interruptions=True))
# ...
# Whenever is appropriate, switch LLMs!
await task.queue_frames([ManuallySwitchServiceFrame(service=llm_google)])
```
- Added an `LLMService.run_inference()` method to LLM services to enable direct, out-of-band (i.e. out-of-pipeline) inference.
### Changed
- Updated `daily-python` to 0.19.8.
- `PipelineTask` now waits for `StartFrame` to reach the end of the pipeline before pushing any other frames.
- Updated `CartesiaTTSService` and `CartesiaHttpTTSService` to align with Cartesia's changes for the `speed` parameter. It now takes only an enum of `slow`, `normal`, or `fast`.
- Added support to `AWSBedrockLLMService` for setting authentication credentials through environment variables.
- Updated `SarvamTTSService` to use WebSocket streaming for real-time audio generation with multiple Indian languages, with HTTP support still available via `SarvamHttpTTSService`.
### Fixed
- Fixed an RTVI issue that was causing frames to be pushed before pipeline was properly initialized.
- Fixed some `get_messages_for_logging()` that were returning a JSON string instead of a list.
- Fixed a `DailyTransport` issue that prevented DTMF tones from being sent.
- Fixed a missing import in `SentryMetrics`.
- Fixed `AWSPollyTTSService` to support AWS credential provider chain (IAM roles, IRSA, instance profiles) instead of requiring explicit environment variables.
- Fixed a `CartesiaTTSService` issue that was causing the application to hang after Cartesia's 5 minutes timed out.
- Fixed an issue preventing `SpeechmaticsSTTService` from transcribing audio.