How to Build an AI Avatar Kiosk for Accessibility with LiveKit and WebSocket
Build an accessible AI avatar kiosk with LiveKit, WebRTC media, WebSocket control, and secure backend sessions.
Introduction
If you are building an accessibility kiosk, the core problem is not “add a chatbot.” It is to create a front end that a person can approach, understand immediately, and use without a keyboard, mouse, or a lot of reading. In practice that means a voice-first experience with a visible, synchronized face that helps with turn-taking, attention, and trust.
That’s a harder systems problem than it looks. You need low-latency speech input and output, a UI that can run unattended, a session model that survives network hiccups, and an avatar pipeline that stays visually in sync with the agent’s speech. By the end of this post, you should be able to sketch a kiosk architecture, understand where WebRTC and WebSocket fit, and wire a realtime avatar into a voice agent without exposing credentials in the browser.
Start with the interaction model, not the avatar
For accessibility, the avatar is not the product. It is the affordance. The actual interaction loop should be simple:
The kiosk listens for speech or accepts a push-to-talk trigger.
Audio is streamed to a voice agent.
The agent returns text/audio with low latency.
The avatar animates from the same speech stream so lip sync stays believable.
The kiosk shows just enough state for the user to recover if anything fails.
This distinction matters because the avatar and the conversational agent are coupled, but they are not the same component. The agent handles ASR, reasoning, and TTS. The avatar layer consumes timing and speech output so it can render mouth motion, blink, gaze, and speaking state. If those layers drift apart, users notice immediately.
For kiosks, I recommend a conservative interaction policy:
Use push-to-talk or an explicit start button rather than open-mic by default.
Show a large visual state indicator: idle, listening, thinking, speaking, error.
Keep response text concise and optionally render transcripts for confirmation.
Design for rapid session reset. Kiosks are shared devices.
Why WebRTC for media, and where WebSocket fits
There are usually two transports in play.
WebRTC is the right tool for low-latency audio/video media between the browser kiosk and your realtime service. It gives you jitter buffering, congestion control, and a transport model built for interactive media. For an avatar kiosk, this is what carries the live voice stream and the synchronized video face. If the system is healthy, users should not be waiting on a full request/response cycle; they should be in a continuous conversational stream.
WebSocket is better for session control and state updates. Think of it as your coordination channel: start session, update instructions, push kiosk state, monitor disconnects, and receive lightweight events. It is not a replacement for the media plane. You do not want to push raw audio/video over a browser WebSocket unless you have a very specific reason to do so.
A good kiosk architecture separates those concerns:
That separation also helps with accessibility. If the media path glitches, your control plane can still tell the user what happened and let them retry without reloading the whole kiosk.
Implement the kiosk shell first
The kiosk itself should be boring infrastructure: full-screen, resilient, and easy to reset. A few implementation details make a big difference:
No shared browser state. Clear local session state on timeout or reset.
Explicit start/stop. Kiosks need a visible affordance for beginning and ending a conversation.
Audio permissions. Plan for the browser permission prompt; do not hide it behind custom UI.
Fallback UI. If video fails, keep voice-only mode available.
If you are managing the session from your own backend, the backend should mint short-lived credentials or session metadata and never ship long-lived API keys to the browser. For a kiosk, that boundary is non-negotiable.
How to wire a voice agent to a synchronized avatar
If your agent already runs in LiveKit, the cleanest path is to add the avatar at the agent layer so the talking face is synchronized with the same speech turn the user hears. That keeps timing aligned and avoids a second “best guess” animation loop in the browser.
In Python, the integration looks like a plugin attached to the agent. Exact configuration fields depend on your docs and avatar/session setup, but the shape is straightforward:
The practical benefit is that the avatar is driven from the same realtime agent pipeline, which usually gives you better lip sync than trying to animate a face from separate browser events. If your voice agent already sits on LiveKit, this is the least invasive integration path.
There is also a Pipecat integration if that is your stack; the project and usage notes are in the repo and guide, but the same principle applies: keep the media and the avatar in the same realtime pipeline so the synchronization problem is solved once.
Session creation and secure control from the backend
For kiosk deployments, the backend usually does three things: creates a session, applies kiosk-specific configuration, and returns only the minimum data the browser needs. That is where the REST API or SDK fits.
Here is an illustrative cURL request pattern against the API. The exact request body fields depend on the endpoint shape in the docs, but this shows the expected authentication style:
In Python, the same pattern is better expressed as a small backend helper that creates sessions and returns an ephemeral token or session payload to the kiosk app:
The key implementation rule is simple: the browser should never see your long-lived API key. For an unattended kiosk, also enforce time limits and IP-based policy server-side so abandoned sessions do not accumulate.
WebSocket control for kiosk state and recovery
In a real deployment, you want a small control channel to keep the kiosk honest. A WebSocket is a good fit for state updates such as “session initialized,” “user speaking,” “agent responding,” “network degraded,” and “session ended.” This lets your UI react quickly without polling.
Keep the messages small and idempotent. A kiosk should tolerate duplicate events and reconnects. The UI should always be able to recover to a known state after a network split.
Where Protoface fits in this kiosk architecture
This is the point where Protoface is useful: it gives you the avatar layer without making you invent a custom lip-sync pipeline. If you are already on LiveKit, the plugin approach is the most natural because the avatar becomes part of the agent itself rather than an independent browser animation. That reduces latency mismatches and keeps the browser simpler.
If you need to manage sessions directly, the REST API and Python SDK let your backend create avatars and realtime sessions, while the dashboard helps you inspect usage and test the flow in a browser before you ship it. For most kiosk teams, the useful pattern is: provision on the backend, stream media through the realtime stack, and keep the browser as thin as possible. The implementation details and current request shapes are in the docs.
Gotchas that matter in production
A few things usually fail first:
Audio echo and feedback. Kiosks often need strict speaker/mic placement and echo cancellation tuned for the room.
Session leaks. If a user walks away, reclaim the session aggressively.
Permission fatigue. Repeated microphone prompts destroy usability; design the kiosk so permissions are granted once and reused safely within policy.
Latency spikes. Keep the visual state honest; if the agent is thinking, say so.
Accessibility beyond speech. Provide transcripts, readable controls, and a manual reset path for staff.
Also remember that “realtime” is not just about low average latency. It is about predictable turn-taking. Users forgive a short delay if the UI clearly indicates that the system is listening or thinking. They do not forgive a face that keeps moving after the agent has stopped speaking.
Conclusion
An accessible AI avatar kiosk is mostly a systems integration problem: keep media realtime, keep control state explicit, and keep secrets off the client. WebRTC belongs on the media path, WebSocket belongs on the control path, and the avatar should be synchronized with the agent rather than animated independently in the browser.
If you are building this stack, start with a thin kiosk shell and a backend that provisions short-lived sessions. Then add the avatar where it belongs in your voice pipeline. For implementation details, integration examples, and current API shapes, go to docs.protoface.com and the relevant GitHub quickstarts.