Introduction

When a voice agent “starts slowly,” the problem is usually not the speech model itself. It’s the end-to-end path: initializing the agent, fetching or generating TTS output, opening the realtime media session, and only then attaching a video face that can lip-sync to the stream. The avatar often becomes visible after the first audio has already started, which makes the whole experience feel laggy even if the underlying model is fast.

This post is about shaving that perceived startup time down in a Python LiveKit agent. By the end, you should be able to identify where avatar latency actually comes from, reduce avoidable setup work, and structure your agent so the face is ready as soon as audio begins.

Where avatar start time really goes

In a typical LiveKit voice agent, there are several distinct startup phases:

• Process and event-loop startup: importing models, loading credentials, constructing the pipeline, and starting the async runtime.

• LLM/TTS warmup: first request latency is often higher than steady-state latency, especially if the provider has to spin up a worker or load a voice preset.

• WebRTC negotiation: the agent joins the room, exchanges SDP, and establishes media tracks.

• Avatar attachment: the video face subscribes to the audio stream and begins generating synchronized frames.

The mistake is treating “avatar start time” as one number. It’s usually an accumulation of small avoidable delays. You can’t eliminate network RTT or model inference, but you can eliminate serial work and redundant initialization.

Start the media path before you need it

The most effective optimization is simple: create the avatar and join the room before the first user utterance is ready to play. In practical terms, you want the agent to have an active audio/video pipeline waiting for the first synthesized chunk, rather than attaching the face after the first sentence has already been generated.

That means:

1. Instantiate the agent and avatar client as early as possible.

2. Connect to LiveKit immediately on session start.

3. Warm up your TTS or speech pipeline with a tiny silent or cached request if your stack supports it.

4. Keep the avatar session alive for the full conversation rather than creating and destroying it per turn.

The key principle is to overlap independent work. If room connection, model warmup, and avatar session creation can run concurrently, do that. Serial startup is the enemy.

Make Python initialization cheap and predictable

In Python, startup overhead often comes from doing too much at import time or re-creating clients repeatedly. Treat the agent process as a long-lived service, not a script that can afford to rebuild everything on every invocation.

A few practical rules:

• Construct SDK clients once per process.

• Reuse HTTP sessions and auth objects where possible.

• Avoid loading large config blobs or model assets inside hot paths.

• Keep avatar/session creation in async code so it can overlap with room join and TTS setup.

Here’s a minimal pattern for a LiveKit voice agent using the Protoface plugin. The exact avatar/session fields depend on your docs, but the shape is what matters: initialize once, attach early, and keep the session alive.

from livekit.agents import WorkerOptions, cli

from livekit.agents import WorkerOptions, cli

from livekit.agents import WorkerOptions, cli

If your current code creates the avatar only after the first response text is generated, you’re guaranteeing a visible delay. Move that work earlier.

Reduce first-turn latency in the speech pipeline

For a talking avatar, the first visible frame depends on the first audio frame. If TTS is slow, the avatar is slow. That makes the speech pipeline the critical path, not the video renderer.

There are three common ways to improve this:

• Pre-warm the TTS provider: make a short startup call so the provider’s cold path happens before the user waits.

• Stream audio instead of buffering: if your stack supports chunked synthesis, begin playback on the first chunk instead of waiting for the full response.

• Use low-latency voices/presets: some voices are more expensive to synthesize than others; measure the trade-off.

Also be careful with prompt design. Long system prompts or retrieval steps before the first token can delay the first audio chunk even when TTS itself is fast. If you need a greeting, keep the first assistant turn short and deterministic. A one-sentence opener is much easier to start quickly than a generated paragraph.

Avoid per-turn avatar creation

One of the most common anti-patterns is creating a new avatar session for every turn in the conversation. That may look clean in code, but it adds unnecessary negotiation, setup, and synchronization overhead. It also makes the UI feel unstable because the face may flicker or reinitialize between turns.

Instead, treat the avatar as part of the session state. If the user remains in the same conversation, keep the avatar alive and just feed it new audio. Reuse the existing media path unless you have a strong reason to tear it down.

That matters even more in multi-step agents where you may chain tool calls, retrieval, and multiple TTS responses. If the avatar is stable across the whole exchange, users perceive the system as much faster even when the backend latency is unchanged.

Instrument the right timestamps

If you want to improve startup time, measure it before optimizing it. The useful timestamps are:

• Process start

• Room join start / complete

• Avatar session create / attach complete

• First assistant token

• First audio chunk sent

• First video frame visible

Once you have those markers, the bottleneck becomes obvious. If room join is slow, look at signaling and network setup. If first audio is slow, focus on LLM/TTS warmup. If video lags behind audio, the avatar attachment path is the likely culprit.

In Python, simple monotonic timestamps are usually enough for a first pass:

import time

import time

import time

Don’t optimize based on intuition. A lot of “avatar latency” turns out to be request latency upstream.

How Protoface fits in

Protoface is useful here because it gives you a dedicated avatar layer that plugs into the existing LiveKit voice-agent flow instead of forcing you to build the video face yourself. In practice, that means you can attach a synchronized talking avatar through the LiveKit plugin and keep the session aligned with your audio stream rather than bolting on a separate video system.

If you’re using the Python integration, the plugin and examples are the fastest way to see the startup path in a real agent. The repository is here: GitHub repo. For direct SDK access or custom session management, the Python SDK is also available, but the same startup principles apply: create the session early, keep it alive, and avoid serializing media setup behind your first response.

If you want to inspect the API surface or confirm the exact fields for session creation, the docs are the right source of truth: docs.protoface.com.

Concrete startup checklist

If you’re tuning a Python LiveKit voice agent today, this is the order I’d look at things:

1. Move avatar/session initialization out of the first-turn handler.

2. Join the LiveKit room as early as possible.

3. Warm up TTS or any upstream model before the first visible response.

4. Stream the first audio chunk instead of buffering the whole answer.

5. Keep the avatar session persistent across the conversation.

6. Add timing logs so you can tell whether the bottleneck is room join, speech synthesis, or avatar attachment.

That sequence is usually enough to remove the “why does the face appear late?” feeling from a voice agent without changing the core product behavior.

Conclusion

Cutting avatar start time is mostly about systems thinking, not a single magic API call. The fastest agents overlap startup work, reuse long-lived sessions, and keep the avatar attached to the media path before the first spoken word goes out.

If you’re building this in Python with LiveKit, start by instrumenting the path from room join to first visible frame, then eliminate unnecessary serialization in your agent startup. For implementation details, examples, and exact request shapes, check the docs at docs.protoface.com and the LiveKit integration examples in the relevant GitHub repo.