v0.0.62
pipecat-ai/pipecatv0.0.62Apr 2, 2025by aconchillo
AI Summary
A major update that reorganizes the service package structure, introduces several new services like Qwen and Piper, and optimizes audio chunking for performance.
Key Highlights
- Reorganized Pipecat services into modular packages (llm, stt, tts, etc.).
- Added SmallWebRTCTransport for P2P WebRTC communication.
- Added WhisperSTTServiceMLX for Apple Silicon and PiperTTSService for TTS.
Breaking Changes
- Deprecated old service import paths, new paths are pipecat.services.[service].[module].
- FunctionCallResultFrame are now system frames to prevent discarding during interruptions.
New Features
- QwenLLMService
- Mem0MemoryService
- WhisperSTTServiceMLX
- PiperTTSService
- SmallWebRTCTransport
Full Release Notes
### Added
- Added `TransportParams.audio_out_10ms_chunks` parameter to allow controlling the amount of audio being sent by the output transport. It defaults to 2, so 20ms audio chunks are sent.
- Added `QwenLLMService` for Qwen integration with an OpenAI-compatible interface. Added foundational example `14q-function-calling-qwen.py`.
- Added `Mem0MemoryService`. Mem0 is a self-improving memory layer for LLM applications. Learn more at: https://mem0.ai/.
- Added `WhisperSTTServiceMLX` for Whisper transcription on Apple Silicon. See example in `examples/foundational/13e-whisper-mlx.py`. Latency of completed transcription using Whisper large-v3-turbo on an M4 macbook is ~500ms.
- `GladiaSTTService` now have comprehensive support for the latest API config options, including model, language detection, preprocessing, custom vocabulary, custom spelling, translation, and message filtering options.
- Added `SmallWebRTCTransport`, a new P2P WebRTC transport.
- Created two examples in `p2p-webrtc`:
- **video-transform**: Demonstrates sending and receiving audio/video with `SmallWebRTCTransport` using `TypeScript`. Includes video frame processing with OpenCV.
- **voice-agent**: A minimal example of creating a voice agent with `SmallWebRTCTransport`.
- Added support to `ProtobufFrameSerializer` to send the messages from `TransportMessageFrame` and `TransportMessageUrgentFrame`.
- Added support for a new TTS service, `PiperTTSService`.
(see https://github.com/rhasspy/piper/)
- It is now possible to tell whether `UserStartedSpeakingFrame` or `UserStoppedSpeakingFrame` have been generated because of emulation frames.
### Changed
- `FunctionCallResultFrame`a are now system frames. This is to prevent function call results to be discarded during interruptions.
- Pipecat services have been reorganized into packages. Each package can have one or more of the following modules (in the future new module names might be needed) depending on the services implemented:
- image: for image generation services
- llm: for LLM services
- memory: for memory services
- stt: for Speech-To-Text services
- tts: for Text-To-Speech services
- video: for video generation services
- vision: for video recognition services
- Base classes for AI services have been reorganized into modules. They can now be found in `pipecat.services.[ai_service,image_service,llm_service,stt_service,vision_service]`.
- `GladiaSTTService` now uses the `solaria-1` model by default. Other params use Gladia's default values. Added support for more language codes.
### Deprecated
- All Pipecat services imports have been deprecated and a warning will be shown when using the old import. The new import should be `pipecat.services.[service].[image,llm,memory,stt,tts,video,vision]`. For example, `from pipecat.services.openai.llm import OpenAILLMService`.
- Import for AI services base classes from `pipecat.services.ai_services` is now deprecated, use one of `pipecat.services.[ai_service,image_service,llm_service,stt_service,vision_service]`.
- Deprecated the `language` parameter in `GladiaSTTService.InputParams` in favor of `language_config`, which better aligns with Gladia's API.
- Deprecated using `GladiaSTTService.InputParams` directly. Use the new `GladiaInputParams` class instead.
### Fixed
- Fixed a `FastAPIWebsocketTransport` and `WebsocketClientTransport` issue that would cause the transport to be closed prematurely, preventing the internally queued audio to be sent. The same issue could also cause an infinite loop while using an output mixer and when sending an `EndFrame`, preventing the bot to finish.
- Fixed an issue that could cause the `TranscriptionUpdateFrame` being pushed because of an interruption to be discarded.
- Fixed an issue that would cause `SegmentedSTTService` based services (e.g. `OpenAISTTService`) to try to transcribe non-spoken audio, causing invalid transcriptions.
- Fixed an issue where `GoogleTTSService` was emitting two `TTSStoppedFrames`.
### Performance
- Output transports now send 40ms audio chunks instead of 20ms. This should improve performance.
- `BotSpeakingFrame`s are now sent every 200ms. If the output transport audio chunks are higher than 200ms then they will be sent at every audio chunk.
### Other
- Added foundational example `37-mem0.py` demonstrating how to use the `Mem0MemoryService`.
- Added foundational example `13e-whisper-mlx.py` demonstrating how to use the `WhisperSTTServiceMLX`.