elevenlabs stt support including scribe v2 realtime#829
elevenlabs stt support including scribe v2 realtime#829simllll wants to merge 7 commits intolivekit:mainfrom
Conversation
|
also merge README
ed81240 to
43b0dd1
Compare
|
It works quite fine, but I have one weird issue, the elevenlabs WS is closing after quite a short time of inactivity. I tried to find a way to add a keepalive, but couldn't find anything that works (there is no keepalive message type and also i cannot just send an empty string). I guess it is related to some kind of timeout/missing keepalive, but as there is no "real" official api documentation right now for this endpoint (except for this doc https://elevenlabs.io/docs/cookbooks/speech-to-text/streaming), I guess we have to either live with it or find another trick ;-) let's see what python implemetation does livekit/agents#3909 code 1000 seems to be not even really an error. |
unfortunately it doens't help the disconnect error. [16:30:14.475] ERROR (390804): STT: WebSocket closed unexpectedly with code 1000: [16:30:14.475] WARN (390804): STT: failed to connect to ElevenLabs, retrying in 0 seconds: Error: WebSocket closed (1/32)

Description
Implement elevenlabs STT support, based on python's elevenlabs and deepgram pattern for relatime in ts.
todo:
Changes Made
Pre-Review Checklist
Testing
Manually on my local machine, tested elevenlabs realtime scribe v2 mostly for now.
restaurant_agent.tsandrealtime_agent.tswork properly (for major changes)Additional Notes
Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.