v0.0.52
pipecat-ai/pipecatv0.0.52Dec 24, 2024by aconchillo
AI Summary
This release adds smart turn detection, DailyTransport SIP call transfer functionality, Koala noise reduction, CerebrasLLMService, and Python 3.13 support.
Key Highlights
- Added smart turn detection example leveraging Gemini 2.0 capabilities
- Added DailyTransport SIP call transfer and DTMF functionality
- Added KoalaFilter for on-device noise reduction
- Added CerebrasLLMService for Cerebras integration
- Added Python 3.13 support with audioop-lts package
Breaking Changes
- AWSTTSService is deprecated, use PollyTTSService instead
New Features
- Timestamped conversation transcript support with TranscriptProcessor
- Expanded language support for ElevenLabs and PlayHT
- Room expiration is now optional in DailyRoomProperties
- Updated DeepgramSTTService to send finalize event on VAD detection
Full Release Notes
### Added - Constructor arguments for GoogleLLMService to directly set tools and tool_config. - Smart turn detection example (`22d-natural-conversation-gemini-audio.py`) that leverages Gemini 2.0 capabilities (). (see https://x.com/kwindla/status/1870974144831275410) - Added `DailyTransport.send_dtmf()` to send dial-out DTMF tones. - Added `DailyTransport.sip_call_transfer()` to forward SIP and PSTN calls to another address or number. For example, transfer a SIP call to a different SIP address or transfer a PSTN phone number to a different PSTN phone number. - Added `DailyTransport.sip_refer()` to transfer incoming SIP/PSTN calls from outside Daily to another SIP/PSTN address. - Added an `auto_mode` input parameter to `ElevenLabsTTSService`. `auto_mode` is set to `True` by default. Enabling this setting disables the chunk schedule and all buffers, which reduces latency. - Added `KoalaFilter` which implement on device noise reduction using Koala Noise Suppression. (see https://picovoice.ai/platform/koala/) - Added `CerebrasLLMService` for Cerebras integration with an OpenAI-compatible interface. Added foundational example `14k-function-calling-cerebras.py`. - Pipecat now supports Python 3.13. We had a dependency on the `audioop` package which was deprecated and now removed on Python 3.13. We are now using `audioop-lts` (https://github.com/AbstractUmbra/audioop) to provide the same functionality. - Added timestamped conversation transcript support: - New `TranscriptProcessor` factory provides access to user and assistant transcript processors. - `UserTranscriptProcessor` processes user speech with timestamps from transcription. - `AssistantTranscriptProcessor` processes assistant responses with LLM context timestamps. - Messages emitted with ISO 8601 timestamps indicating when they were spoken. - Supports all LLM formats (OpenAI, Anthropic, Google) via standard message format. - New examples: `28a-transcription-processor-openai.py`, `28b-transcription-processor-anthropic.py`, and `28c-transcription-processor-gemini.py`. - Add support for more languages to ElevenLabs (Arabic, Croatian, Filipino, Tamil) and PlayHT (Afrikans, Albanian, Amharic, Arabic, Bengali, Croatian, Galician, Hebrew, Mandarin, Serbian, Tagalog, Urdu, Xhosa). ### Changed - `PlayHTTTSService` uses the new v4 websocket API, which also fixes an issue where text inputted to the TTS didn't return audio. - The default model for `ElevenLabsTTSService` is now `eleven_flash_v2_5`. - `OpenAIRealtimeBetaLLMService` now takes a `model` parameter in the constructor. - Updated the default model for the `OpenAIRealtimeBetaLLMService`. - Room expiration (`exp`) in `DailyRoomProperties` is now optional (`None`) by default instead of automatically setting a 5-minute expiration time. You must explicitly set expiration time if desired. ### Deprecated - `AWSTTSService` is now deprecated, use `PollyTTSService` instead. ### Fixed - Fixed token counting in `GoogleLLMService`. Tokens were summed incorrectly (double-counted in many cases). - Fixed an issue that could cause the bot to stop talking if there was a user interruption before getting any audio from the TTS service. - Fixed an issue that would cause `ParallelPipeline` to handle `EndFrame` incorrectly causing the main pipeline to not terminate or terminate too early. - Fixed an audio stuttering issue in `FastPitchTTSService`. - Fixed a `BaseOutputTransport` issue that was causing non-audio frames being processed before the previous audio frames were played. This will allow, for example, sending a frame `A` after a `TTSSpeakFrame` and the frame `A` will only be pushed downstream after the audio generated from `TTSSpeakFrame` has been spoken. - Fixed a `DeepgramSTTService` issue that was causing language to be passed as an object instead of a string resulting in the connection to fail.