What Is a Realtime AI Avatar for Employee Onboarding? A Developer’s Guide to the Architecture
Developer guide to realtime AI avatar architecture for employee onboarding: latency, WebRTC, LiveKit, REST APIs, embeds, and session security.
Introduction
If you are building an onboarding flow with a voice agent, the obvious next step is usually “give it a face.” A realtime AI avatar is the piece that turns a spoken conversation into a synchronized audiovisual interaction: the agent listens, reasons, responds, and shows a lip-synced video face with low enough latency that the interaction feels continuous instead of staged.
For employee onboarding, that matters because the avatar is not just decoration. It changes how users perceive turn-taking, confidence, and responsiveness. The practical question for developers is how to wire this up without adding brittle media plumbing to your product.
This guide walks through the architecture of a realtime avatar for onboarding, the latency and transport decisions that matter, and the integration patterns you will actually use in production. By the end, you should be able to reason about where the avatar sits in your stack, how it connects to a voice agent, and what trade-offs show up in a web app versus a backend-driven deployment.
What “realtime avatar” means in practice
At the implementation level, a realtime avatar is a media endpoint that takes conversational input and emits synchronized video frames, usually with audio or timing metadata that keeps lip movement aligned to speech. In an onboarding flow, the avatar typically sits downstream from a voice agent:
The user speaks into a browser or app.
Speech is transcribed by your ASR/voice stack.
An LLM generates the response.
TTS produces audio for the reply.
The avatar renders a face that matches that audio closely enough for the interaction to feel live.
The key point is that the avatar is not the agent itself. It is a media layer attached to the agent. That distinction matters for architecture because the agent can be stateful and deterministic while the avatar is an output surface with its own timing constraints.
Architecture: where the latency actually comes from
For onboarding, your target is not cinematic quality; it is perceptual continuity. Users tolerate some delay, but they are very sensitive to mismatched mouth movement, audio drift, or long pauses between turns. A useful mental model is to budget latency across three stages:
Input latency: microphone capture, network transport, and ASR.
Reasoning latency: orchestration, retrieval, LLM generation, and policy checks.
Output latency: TTS synthesis, video generation, and frame delivery.
The avatar layer should minimize output latency and preserve synchronization. In most deployments, that means using a realtime transport such as WebRTC for live sessions, because it gives you low-latency media streams and handles jitter better than a pure polling-based design. The avatar renderer typically receives audio timing cues or streaming audio chunks and generates corresponding frames continuously. If your media pipeline buffers too much, the face will look “behind” the voice; if it buffers too little, the motion becomes unstable under network jitter.
For onboarding flows, this also affects UX decisions. You usually want:
short turn-taking windows
barge-in support if the user interrupts
explicit thinking states for longer answers
predictable fallback behavior when media degrades
These are product decisions, but they are constrained by the media architecture underneath.
Integration pattern: voice agent plus avatar
The cleanest architecture is to keep the agent logic in your existing voice stack and attach the avatar as a presentation layer. In a LiveKit-based system, that usually means the agent emits audio as it already does, and a plugin handles avatar synchronization.
One practical advantage of this pattern is that you do not need to rewrite your conversational orchestration. Your agent can keep using the same state machine, tool calls, or retrieval logic; the avatar only needs access to the realtime session and the generated speech stream.
A minimal Python example looks like this:
The important thing is not the exact method name, which may differ by framework, but the boundary: the agent owns conversation state, while the avatar owns presentation. That separation keeps your onboarding logic testable and makes it easier to swap models, TTS providers, or rendering quality tiers without changing your product flow.
Backend-managed sessions: when the server should own the media state
In onboarding flows, you often need more than a single live conversation. You may want a session record, per-user identity, rate limits, or the ability to initialize an avatar before the front end connects. That is where a backend API becomes useful.
A typical server-side flow is:
Create or select an avatar configuration.
Start a realtime session for the employee.
Pass the session token or embedded session URL to the client.
Track usage, timeout, and cleanup on the backend.
Here is an illustrative cURL call against the REST API; exact request fields depend on the endpoint documented in the docs:
For a developer platform, this server-managed model is usually the right default. It keeps API keys off the browser, allows backend enforcement of duration limits, and makes it easier to associate sessions with employee records or onboarding milestones.
Browser embeds and security trade-offs
Sometimes the simplest onboarding experience is just an interactive avatar embedded on a page. In that case, an iframe-based embed is the cleanest option because it removes most of the client-side integration burden and avoids exposing secrets in the browser.
The security model matters here. A customer-managed iframe embed can be configured so the parent origin must be allowlisted, the voice and instructions are scoped per embed, and rate limits can be enforced per IP and duration. Those controls are important if you are placing an onboarding assistant in a public-facing app or internal portal where you still want strong boundaries.
The trade-off is that iframe embeds simplify integration but reduce direct control over the media pipeline from the host page. That is usually fine for onboarding, where reliability and safety matter more than deep UI coupling. If you need tight coordination with your own page state, use the REST API or backend session management instead.
How Protoface fits without changing your architecture
Protoface is useful here because it slots into the avatar layer rather than forcing you to rebuild the rest of the stack. If you already have a voice agent, the LiveKit plugin lets you add a synchronized talking face with minimal surface area. If you want server control, the REST API and Python SDK let you create avatars and sessions programmatically. If you want a browser-first deployment, the iframe embed keeps keys and media policy off the client.
In practice, that means you can choose the integration point based on your product constraints:
LiveKit agent: best when you already have a realtime voice pipeline.
REST API / Python SDK: best when onboarding sessions are created by your backend.
Iframe embed: best when you want the fastest path to an interactive web experience.
If you want a reference implementation, the quickstarts linked from the project repo are the fastest way to see the media/session flow end to end, and the public docs at docs.protoface.com are where the concrete request shapes live.
Implementation gotchas worth planning for
There are a few failure modes that show up repeatedly in onboarding systems:
Model latency spikes: if the LLM stalls, do not freeze the avatar. Show a brief “thinking” state or hold a neutral pose.
Audio/video drift: if you buffer audio too aggressively, lip sync will lag. Keep the timing source close to the playback pipeline.
Session cleanup: always close sessions when the user finishes onboarding, times out, or disconnects.
Policy mismatches: onboarding scripts often include sensitive instructions; keep them server-side when possible.
Quality-tier cost: higher visual fidelity is not free. Pick the tier that matches the product surface, not the highest tier by default.
The last point is important. For internal onboarding, a slightly lower visual quality tier may be perfectly acceptable if it reduces cost and improves consistency. Users care far more about responsiveness and speech alignment than about photorealism.
Conclusion
A realtime AI avatar for employee onboarding is, architecturally, a media layer attached to a voice agent. The core engineering problems are latency control, synchronization, session management, and security boundaries. Once you separate agent logic from avatar rendering, the implementation becomes much easier to reason about and test.
If you are building this now, start by deciding where the session should live: in your LiveKit agent, in your backend via REST/Python, or in an iframe embed. Then wire up one end-to-end path and measure turn latency and lip sync before you optimize anything else.
For implementation details, examples, and the exact API shapes, start with the docs and the relevant quickstarts from the project repos.