Introduction

If you are building a virtual receptionist, the hard part is not just generating a good answer. It is delivering that answer as a low-latency, lip-synced avatar response that feels like a single coherent turn: user speech ends, the model thinks, the avatar starts speaking, and the mouth movement stays aligned with the audio stream.

This post walks through the practical architecture for doing that in vanilla JavaScript. By the end, you should understand how to wire together browser media capture, a realtime transport, TTS or a voice agent, and an avatar video stream so the result is usable in production rather than just demo-friendly.

What “lip-synced streaming” actually means

For a receptionist, the avatar usually does not render a full precomputed video clip. Instead, you stream audio and video as the response is being generated. The important constraint is synchronization:

• The audio must arrive early enough to keep latency low.

• The avatar video must be generated from the same utterance, not from a separate timer.

• The client must handle reconnects, buffering, and turn boundaries without visually “popping” the face between states.

In practice, this means your frontend should treat the avatar as a realtime media surface, not as a static image or a set of discrete animations. If you try to fake lip sync locally from audio amplitude, you will usually get motion that looks plausible but does not track phonemes well enough for a receptionist experience.

Browser architecture in vanilla JS

The cleanest setup is:

1. Capture the visitor’s microphone input in the browser.

2. Send that audio to your voice agent or conversational backend.

3. Receive a streaming assistant response as audio, plus a synchronized avatar video track or avatar session stream.

4. Render the avatar stream into a <video> element, and optionally route the audio to the same media pipeline.

For a receptionist, the browser code should stay thin. You want your JS to manage session lifecycle, attach media tracks, and handle UI state. The actual turn-taking logic, transcription, response generation, and avatar synthesis should live behind an API or realtime service.

Using WebRTC without overcomplicating the client

Most realtime avatar implementations end up on WebRTC for the media path because it gives you low latency, jitter handling, and a standard way to carry audio/video tracks. In the browser, the main job is to connect to the remote session and attach the incoming video track to a visible element.

A minimal pattern looks like this:

// Pseudocode: exact connection/session fields depend on your backend or avatar provider.

// Pseudocode: exact connection/session fields depend on your backend or avatar provider.

// Pseudocode: exact connection/session fields depend on your backend or avatar provider.

The important thing here is not the exact signaling code; it is the separation of responsibilities. The browser does capture and playback. The server or avatar platform decides when the assistant is speaking and generates the synchronized facial motion.

Two implementation details matter a lot in production:

• Autoplay policy: browsers may block playback until the user interacts with the page. Start the session from a click or tap.

• Audio output handling: if the avatar audio and video arrive separately, make sure they are attached to the same session lifecycle so the user does not hear the response before the face starts moving.

Turn-taking and latency budget

The biggest source of “this feels fake” is not raw model quality. It is latency. A receptionist can tolerate a short thinking pause, but the transition from user silence to assistant speech has to feel intentional.

In practice, budget for these phases:

• End-of-speech detection: decide when the user is done talking.

• Inference: generate the assistant response or TTS text.

• Media startup: get the first audio and the first video frames flowing.

Keep your browser state machine simple:

idle → listening → thinking → speaking → listening

idle → listening → thinking → speaking → listening

idle → listening → thinking → speaking → listening

Do not start and stop the avatar stream for every partial transcript. Wait until you have a stable turn boundary. Otherwise, you get jittery handoffs and unnecessary session churn.

Rendering the avatar cleanly in the DOM

Once the stream is attached, the UI work is mostly about avoiding visual defects. For a receptionist widget, a solid default is a fixed aspect-ratio container with the video filling the frame. If the avatar background is transparent or keyed, keep the container styling consistent so layout shifts do not distract users.

<div class="avatar-shell">
</div>

<div class="avatar-shell">
</div>

<div class="avatar-shell">
</div>

A few practical notes:

• playsinline matters on mobile Safari.

• muted is often required to satisfy autoplay restrictions during setup; you can unmute only if your design needs local playback.

• Use CSS object-fit: cover if you want the face to fill the frame without letterboxing.

If you are showing captions or a transcript alongside the avatar, update those independently of video frame rate. The transcript should follow the text stream; the video should just mirror the speech. Coupling both to the same render loop usually makes the UI worse.

Where Protoface fits

This is exactly the kind of integration Protoface is meant for: you keep your vanilla JS client focused on media handling, while the avatar service handles the synchronized face generation. For browser-first apps, the customer-managed iframe embed is the lowest-friction path because you do not expose an API key in the browser and you can control the embed with per-iframe settings and allowlists. If you are already running a voice-agent backend, the REST API at docs.protoface.com is the place to create avatars and realtime sessions, then hand the client a session-specific connection flow.

If your stack is built around Python or LiveKit, there are also server-side integration surfaces that keep the media path in one place rather than forcing the browser to orchestrate everything. The design choice is mostly about where you want the turn logic to live: in your app, in your agent framework, or in an embed.

import requests

print(resp.json())

import requests

print(resp.json())

import requests

print(resp.json())

Failure modes you should design for

Most issues in production are not “the avatar is broken”; they are connectivity and lifecycle bugs. Watch for these:

• Session expiry: your UI should detect when the assistant session is no longer valid and fall back cleanly.

• Network churn: if the connection drops, show a reconnect state rather than freezing the last frame indefinitely.

• Audio desync: if playback gets delayed, it is better to briefly buffer than to show the mouth moving against silence.

• Mobile constraints: camera/mic permissions, autoplay rules, and background tab throttling all affect realtime UX.

Also be disciplined about rate limiting and access control. If your receptionist is public-facing, do not leave privileged session creation logic in the browser. Keep secrets on the server or use an embed architecture that is explicitly designed to avoid exposing keys.

Implementation checklist for a solid receptionist experience

• Start the session from a user gesture.

• Use a realtime transport for media, not polling.

• Keep turn detection on the server or agent side.

• Attach the incoming avatar stream directly to a video element.

• Handle reconnect and session-expiration states explicitly.

• Measure end-to-end latency, not just model latency.

If you are evaluating integration paths, the quickest way to get something working is usually to start from a quickstart and adapt only the pieces you need. The examples linked from the project README are a good reference point for the overall flow, even if your final app is much simpler.

Conclusion

To stream lip-synced AI avatar responses in vanilla JS, treat the avatar as a realtime media session: capture audio, forward it to your backend or agent, and render the synchronized video stream back in the browser. The frontend should stay small and predictable; the hard problems are turn-taking, latency, and lifecycle management.

If you want a practical path from prototype to production, start with the docs, then choose the integration surface that matches your stack: browser embed if you want minimal frontend/backend code, REST if you want full control, or a voice-agent/plugin path if your application already has a conversational runtime. From there, build the UI around clean session state transitions and measure perceived latency from the user’s last word to the avatar’s first spoken frame.