feat(examples): add LemonSlice avatar actions#1688
feat(examples): add LemonSlice avatar actions#1688rosetta-livekit-bot[bot] wants to merge 1 commit into
Conversation
|
| const avatar = makeAvatar(initial); | ||
| const sessionId = await avatar.start(session, ctx.room); | ||
| const state: State = { persona: initial, avatar, sessionId }; | ||
|
|
||
| if (supportsActions(initial.id)) { | ||
| actions.setSession(state.sessionId, initial.id); | ||
| } | ||
|
|
||
| await session.start({ | ||
| agent: makeAgent(initial, actions), | ||
| room: ctx.room, | ||
| }); |
There was a problem hiding this comment.
🔴 Avatar started before AgentSession.start() causes RoomIO to overwrite avatar audio output
The avatar is started at line 90 (await avatar.start(session, ctx.room)) BEFORE session.start() at line 97. The lemonslice AvatarSession.start() sets agentSession.output.audio = new DataStreamAudioOutput(...) (plugins/lemonslice/src/avatar.ts:255). Then when session.start() runs, it creates a RoomIO which unconditionally overwrites agentSession.output.audio with its own ParticipantAudioOutput (agents/src/voice/room_io/room_io.ts:578-580). The _startImpl warning at agents/src/voice/agent_session.ts:497-501 says "ignoring" but doesn't actually prevent the overwrite.
This means the avatar will never receive audio and won't animate on initial startup. All other avatar examples (trugen_avatar.ts, bey_avatar.ts, lemonslice_realtime_avatar.ts) start the avatar AFTER session.start(), so the avatar's start() correctly overrides RoomIO's output. The persona-switch path (lines 137-138) is unaffected since the session is already started and RoomIO doesn't re-run.
| const avatar = makeAvatar(initial); | |
| const sessionId = await avatar.start(session, ctx.room); | |
| const state: State = { persona: initial, avatar, sessionId }; | |
| if (supportsActions(initial.id)) { | |
| actions.setSession(state.sessionId, initial.id); | |
| } | |
| await session.start({ | |
| agent: makeAgent(initial, actions), | |
| room: ctx.room, | |
| }); | |
| await session.start({ | |
| agent: makeAgent(initial, actions), | |
| room: ctx.room, | |
| }); | |
| const avatar = makeAvatar(initial); | |
| const sessionId = await avatar.start(session, ctx.room); | |
| const state: State = { persona: initial, avatar, sessionId }; | |
| if (supportsActions(initial.id)) { | |
| actions.setSession(state.sessionId, initial.id); | |
| } |
Was this helpful? React with 👍 or 👎 to provide feedback.
Summary
actions.py)