Introduction

If you embed a realtime avatar on a vanilla JavaScript site, the first impression is usually determined by two latency budgets, not one: how long the avatar takes to appear, and how long it takes before it can actually speak. Those are related, but they come from different parts of the stack. Rendering and video startup affect perceived load time. Speech synthesis, model warm-up, and transport setup affect first-speech latency.

By the end of this post, you should be able to inspect both budgets separately, reduce avoidable round trips, and structure your client so the avatar can connect, render, and begin speaking with fewer stalls. I’ll also show where Protoface fits when you want to outsource the avatar/session plumbing instead of building it from scratch.

Separate “avatar load” from “time to first speech”

It helps to measure these as distinct milestones:

• Avatar load time: from page interaction or route entry until the user sees the avatar surface and the video element is ready to render frames.

• First-speech latency: from user action or agent start until the first audible audio frame or lip-synced utterance begins.

In a browser app, the avatar can be “loaded” in the DOM long before the media pipeline is ready. Conversely, a WebRTC session can be connected while the avatar still looks frozen because the first composited frame has not arrived yet. Treating those as one metric hides the bottleneck.

A practical way to instrument this is with a few timestamps around the edges of your app:

const t0 = performance.now();

});

const t0 = performance.now();

});

const t0 = performance.now();

});

These numbers tell you which layer to optimize: DOM work, network setup, media negotiation, or agent startup.

Optimize the browser path first

On a vanilla JavaScript site, avoid making the avatar element itself expensive to mount. The fastest path is usually:

1. Render a lightweight placeholder immediately.

2. Load the avatar runtime lazily, after initial content is visible.

3. Connect the realtime session only when the user is likely to interact.

The biggest self-inflicted delays I see are unrelated to avatar transport:

• Blocking the main thread with large JS bundles, heavy state initialization, or synchronous parsing before the avatar code even runs.

• Waiting for user interaction before preconnect when the session could have been initialized earlier in the idle period.

• Repeatedly creating and destroying media elements, which forces the browser to renegotiate decoders and layout.

For a simple site, use an explicit mount point and keep the DOM stable:

<div id="avatar-shell">
</div>

<div id="avatar-shell">
</div>

<div id="avatar-shell">
</div>

async function mountAvatar() {
}

async function mountAvatar() {
}

async function mountAvatar() {
}

If you’re embedding via iframe, the same principle applies: allocate the container up front so the iframe can load without relayout, and avoid re-creating the frame on every state change. That keeps startup predictable and preserves the session while the page updates around it.

Reduce first-speech latency by prewarming the right things

Once the browser path is under control, first-speech latency usually comes from one of four sources:

• Session creation: auth, room/session setup, or avatar allocation.

• Media negotiation: WebRTC signaling, ICE candidate exchange, and transport establishment.

• Agent warm-up: model initialization, prompt loading, or tool registration.

• Speech pipeline startup: TTS synthesis, audio buffering, and lip-sync alignment.

The main optimization is to move work earlier, but only when it won’t waste resources. Some useful patterns:

1) Pre-create the session before the user speaks. If the UI makes it clear that an assistant is available, create the avatar/session as soon as the page or conversation view is ready. You can keep the transport idle until the user begins the interaction. This pays the connection cost before the first utterance is requested.

2) Reuse the connection. If your app supports multiple turns, keep the session alive across turns instead of tearing it down after each response. Reconnection overhead is often larger than the cost of idling for a short period.

3) Keep the first response short. If your agent can greet the user with a short acknowledgment while a longer answer is generated in the background, users perceive the session as responsive even if deeper reasoning continues after the first frame of speech.

4) Minimize browser-side work before the first frame. Don’t attach unnecessary listeners, analytics hooks, or layout measurements in the critical path. It’s common for “just a few extra callbacks” to show up as visible delay on slower devices.

When you measure first-speech latency, break it down. If connect time dominates, focus on session setup and transport. If the gap between connect and speech dominates, focus on the agent and speech synthesis path. If the first video frame lags audio, your render path or decoding path is likely the issue.

Practical client-side tactics for vanilla JavaScript

For a plain browser app, the simplest wins are usually the most effective:

• Prefetch or preload non-sensitive assets that are safe to fetch early, such as app JS chunks or static avatar shell assets.

• Use requestIdleCallback or a low-priority task to initialize the avatar client after the page is interactive.

• Warm the connection on hover, focus, or visible intent rather than waiting for the submit button click.

• Keep a single avatar instance for the lifetime of the conversation view.

Here’s a simple pattern for deferring setup until the user is near the interaction point:

let avatarReady = false;

document.getElementById('chat-input').addEventListener('pointerenter', warmAvatar);

let avatarReady = false;

document.getElementById('chat-input').addEventListener('pointerenter', warmAvatar);

let avatarReady = false;

document.getElementById('chat-input').addEventListener('pointerenter', warmAvatar);

That said, don’t prewarm so aggressively that you create sessions for visitors who never interact. If the avatar is expensive to allocate, use a clear intent signal from the user interface.

What actually helps on the network path

Realtime avatars typically use a persistent transport such as WebRTC so media can flow with low latency and adaptive jitter handling. That is good for turn-taking, but it means you should think about the connection lifecycle like any other realtime system:

• DNS/TLS happen before any signaling can start, so keep your origin and API endpoints stable.

• Signaling should exchange only the data required to establish the session.

• ICE/DTLS setup can vary a lot by network, especially on corporate or mobile networks.

A few rules of thumb:

• Don’t create and destroy transport sessions on every message.

• Avoid large JSON payloads in the critical path; pass the minimum needed to bootstrap the avatar.

• Separate the user-facing “agent is thinking” state from the actual media session state, so UI can update immediately even while media is still coming up.

If your avatar is embedded in an iframe, this is especially important because the iframe adds an origin boundary and its own startup cost. The upside is security and isolation. The trade-off is that you should be deliberate about when the frame is inserted and when the session begins.

Where Protoface fits

If you want the avatar/session layer handled for you, the useful question is not “how do I build video faces?” but “how do I get a session ready with minimal browser code and no API key exposure?” For a vanilla JavaScript site, the iframe embed surface is the cleanest fit: the parent page stays backend-free, the browser never sees a secret, and you can control the interaction surface with allowlists and rate limits on the embed side.

For developers who are already operating a voice agent backend, the REST API and Python SDK are the lower-level surfaces. A common pattern is to create or manage a session server-side, then hand the browser only the short-lived session details it needs to connect. Exact request fields and object shapes are in the docs, but the flow looks like this:

import os

print(session)

import os

print(session)

import os

print(session)

If you are already using a LiveKit voice agent, the plugin path is the most direct way to attach a synchronized face without inventing your own media plumbing. The repository and examples are the fastest way to see how the avatar is injected into the agent flow: GitHub. For Pipecat users, there is also a dedicated integration guide in the package docs.

In all cases, the same latency advice applies: initialize before the user expects a response, keep the session alive across turns, and avoid doing expensive work on the critical path to the first frame of speech.

Conclusion

Reducing avatar load time and first-speech latency is mostly about respecting the browser and media pipeline. Measure the milestones separately, keep the DOM and JS path lightweight, prewarm the session when there is real user intent, and reuse the transport instead of rebuilding it on every turn. If you’re integrating a realtime avatar into a vanilla JavaScript site, the fastest wins usually come from simpler mounting, earlier session setup, and fewer moving parts in the first interaction.

If you want implementation details, API shapes, or integration examples, start with the docs. If you want to see the plugin and quickstarts in context, the GitHub examples are the shortest route from “works in principle” to “works in production.”