What Does a Realtime Employee Onboarding Assistant Actually Need? A Technical Breakdown of Avatar, Voice, and Chat Components
Technical breakdown of realtime onboarding assistants: avatar sync, low-latency voice, chat fallback, and integration patterns.
Introduction
If you are building a realtime employee onboarding assistant, the hard part is not “making it talk.” The hard part is keeping three streams aligned under latency, state, and UX constraints: the avatar that users look at, the voice path that users hear, and the conversation state that drives both.
In practice, an onboarding assistant is a small realtime system. It needs to answer HR questions, walk a new hire through forms, explain policies, and stay visually present while the dialogue changes course. By the end of this post, you should have a clear model for what the avatar, voice, and chat components each do, where the failure modes are, and how to wire them together without building a brittle demo.
Start with the system boundary: what actually has to happen in realtime
An onboarding assistant usually sits on top of a voice or multimodal agent. The core loop looks like this:
1. The user speaks or types a question.
2. Speech is transcribed or text is received by the agent.
3. The agent decides what to say next, possibly calling tools or fetching policy data.
4. The response is synthesized as audio.
5. The avatar renders a talking face that stays synchronized with that audio.
That last step is easy to underestimate. A “video face” is not just decoration; it is a realtime rendering problem with strict timing requirements. If audio starts late, lip sync drifts, or the face freezes while the agent is still thinking, users immediately notice. For onboarding, that matters because the system is often explaining unfamiliar material. Users pay attention to pauses, turn-taking, and whether the assistant looks like it is actually listening.
The three surfaces have different responsibilities:
Chat carries explicit text, fallback interactions, citations, and form-like inputs.
Voice handles turn-taking, latency tolerance, and the conversational feel.
Avatar communicates presence, eye contact, and speech sync, but should not be treated as the source of truth.
If you mix those concerns, debugging becomes painful. For example, a transcript may be correct while the avatar is still finishing an old utterance because the rendering layer did not receive a proper “speech end” signal. Or the avatar may be correct while the chat panel shows a newer response, because tool calls completed out of order. The architecture should make those states explicit.
What the avatar component really needs
The avatar layer has one job: render a convincing synchronized speaker that tracks the current turn. In practical terms, that means you need a media transport that can keep latency low and preserve order. WebRTC is the common fit because it is optimized for realtime audio/video delivery, handles NAT traversal, and supports the low-latency interaction pattern you want for live conversation.
But the avatar is not just video transport. It also needs session state:
Active speaker state so the face starts and stops with the utterance.
Utterance boundaries so lip sync matches each response, not just a stream of audio chunks.
Interrupt handling so the avatar can visibly stop when the user barges in.
Quality controls so you can choose the trade-off between realism, bandwidth, and cost.
For an onboarding assistant, this means you should design for short, clear turns. Long monologues increase the chance of sync drift and make it harder to interrupt with a follow-up question like “What if I don’t have my tax form yet?” If you need longer explanations, chunk them into smaller turns and let the agent yield between chunks.
What the voice layer needs: low-latency turn-taking, not just TTS
Voice is where most systems fail first. Users are very sensitive to turn latency. If the assistant takes too long to respond, it feels less like a live onboarding guide and more like a form with a chatbot wrapper.
The voice path should provide three properties:
Fast commit to speech after the agent decides on a response.
Natural interruption behavior when the user starts talking again.
Stable turn state so downstream components know whether the assistant is listening, thinking, or speaking.
For a new hire experience, interruptions are not edge cases. Users frequently ask clarifying questions mid-answer, especially when they hear terms like “benefits eligibility,” “W-4,” or “direct deposit.” Your agent should not force them to wait for the entire answer before it can react. Instead, treat user speech as a high-priority event that can cancel or truncate the current response.
From an implementation perspective, this usually means your voice agent needs:
speech detection / VAD to identify user turns,
ASR or text input to feed the conversational model,
a response generator that can stream partial output, and
a TTS engine or audio stream that can be cut over cleanly when interrupted.
The avatar must follow the same state machine. If the voice layer cancels a response, the face should stop speaking immediately rather than finish the previous sentence. That alignment is the difference between a system that feels realtime and one that feels loosely coupled.
What the chat layer is for: precision, auditability, and fallback
Chat is often the most reliable surface in the stack, and for onboarding it should do more than mirror speech. It is the place to surface exact policy text, links to forms, and structured follow-up questions that are hard to ask naturally out loud.
Use chat for the things voice is bad at:
reading precise dates, plan names, or IDs,
showing links to documents or internal systems,
collecting structured answers like preferred start date or tax withholding state,
capturing a transcript for audit or review.
That matters because onboarding questions often have compliance implications. A voice-only experience can be friendly, but the user still needs a durable record of what was asked and answered. The chat transcript should be considered the system of record for the interaction, even if the avatar is the primary presentation layer.
A good pattern is to keep the conversational model authoritative and then derive both voice and chat from the same response object. That gives you one source of truth for content, with two renderers:
The exact schema is up to your stack, but the principle is the same: avoid letting the voice generator, avatar renderer, and chat UI invent their own independent versions of the answer.
Integration patterns that work in practice
There are two common ways to build this, depending on whether you are embedding the assistant into an existing voice agent or creating a web-first experience.
1. Voice-agent-first integration
If you already have a LiveKit-based voice agent, the cleanest path is to add a synchronized face via the livekit-plugins-protoface plugin. The plugin approach keeps your conversational logic where it already lives and attaches avatar delivery as a realtime media concern.
Conceptually, the agent produces speech, and the plugin turns that into a visible speaker. The implementation details depend on your agent stack, but the shape is straightforward:
If you are using Pipecat instead of LiveKit directly, the integration path is documented in the Pipecat guide and examples: Protoface video service reference. That is useful when your agent pipeline already has a media graph and you want to drop the avatar into it without inventing a separate transport layer.
2. Web embed for fast deployment
If you need the assistant on an internal onboarding page, an iframe embed is often the least risky option. The main advantage is security and operational simplicity: the browser never sees an API key, and the embed can be constrained with parent-origin allowlists plus per-embed voice and custom instructions. Rate limits by IP and duration are also important here because onboarding pages can be shared broadly inside a company.
That model is a good fit when you want HR or IT to paste an assistant onto a portal without standing up backend code. It is less flexible than a fully custom agent pipeline, but for many onboarding workflows the constraints are exactly what you want.
Common failure modes and how to design around them
The failure modes are usually not dramatic; they are small timing and state issues that add up to a poor experience.
Audio arrives before the avatar is ready. Buffer the first few frames or delay playback until the session is established.
User interrupts, but the avatar keeps talking. Make interruption a first-class state transition in the agent and propagate cancellation downstream.
Chat and voice disagree. Generate both from the same canonical response object, not separate prompts.
The agent over-explains. Keep turns short and let the user drive follow-ups.
The UI hides latency. Show listening/thinking/speaking states explicitly so users know the system is alive.
For onboarding specifically, I would also avoid over-animating the avatar. The goal is clarity, not novelty. A stable face with good lip sync and predictable turn behavior is better than a highly expressive model that occasionally desynchronizes.
How Protoface fits here
Protoface is useful when you want the avatar layer to behave like infrastructure rather than a one-off frontend feature. The REST API is appropriate when you need to create and manage avatars or sessions from your backend. The Python SDK is useful when you want to automate that from application code. And if you already have a voice agent, the LiveKit plugin gives you the shortest path to a synchronized talking face without rewriting the rest of the stack.
For onboarding systems, that usually means one of two things: either attach a face to an existing agent pipeline, or embed a managed avatar into a portal where HR can launch it without exposing credentials. The main thing Protoface contributes is a clean separation between media/session management and your application logic, which is exactly what you want when the assistant has to be reliable under real user traffic.
That request shape is illustrative; check the docs for the exact session fields and lifecycle. The important part is the pattern: create a session server-side, hand the client only what it needs, and keep your API key out of the browser.
Conclusion
A realtime onboarding assistant is not “chat plus a face.” It is a coordinated voice system with explicit state, tight latency requirements, and a UI that has to stay honest when the conversation changes direction. The avatar needs synchronized speech state, the voice layer needs fast interruption and turn management, and the chat layer needs to preserve precision and auditability.
If you keep those responsibilities separate, you can build something that feels genuinely live instead of merely animated. If you want to see the available integration surfaces and examples, start with the docs and the quickstarts linked from the Protoface repository.