Introduction

If you already have a Next.js app with a voice agent, adding a talking face is mostly a systems integration problem: you need low-latency media transport, synchronized audio/video, and a UI that can cleanly embed a realtime session without turning your frontend into a signaling stack.

This post shows a practical way to do that with LiveKit for transport and an avatar layer for synchronized video generation. By the end, you should understand how the pieces fit together, where the latency and synchronization boundaries are, and how to wire the backend and frontend so the avatar speaks in step with your agent instead of lagging behind it.

What actually has to happen

A talking avatar is not just a video overlay. In a realtime voice flow, your agent produces audio incrementally, and the avatar stream must track that audio closely enough that lip movement feels anchored to the speech. In practice that means:

• The agent runs a realtime conversation loop, usually over WebRTC or a similar low-latency media path.

• Audio chunks are emitted as the model speaks.

• An avatar service consumes that audio and renders a synchronized face video stream.

• The client subscribes to both streams and displays them with minimal buffering.

The important engineering constraint is that the avatar should be driven by the same conversation state as the audio. If the avatar is updated independently, you get obvious desynchronization: mouth motion that leads or trails the voice, expressions that don’t match the current utterance, or frame drops when the browser reflows the layout.

Use LiveKit as the transport layer, not the rendering layer

For a Next.js app, the clean split is to let LiveKit handle realtime session transport and agent orchestration, while the avatar component only concerns itself with the video face. That keeps the browser code small and avoids mixing agent logic with media plumbing.

In this setup, your app typically does three things:

1. Creates or joins a LiveKit room for the conversation.

2. Runs your voice agent in the backend.

3. Attaches an avatar stream to that agent so the spoken output has a synchronized face.

From the frontend’s perspective, you subscribe to the room and render the published tracks. The avatar is just another video source, but the backend is responsible for ensuring that source stays aligned with the agent’s speech timing.

Backend wiring: add the avatar to the agent

If you are using a Python-based LiveKit agent, the least invasive integration point is the LiveKit plugin for avatar support. The plugin sits inside the agent process and takes care of producing the synchronized video face from the agent’s output. The exact configuration fields can vary, so treat the snippet below as a shape, not a copy-paste contract.

from livekit import agents

await agent.run(ctx)

from livekit import agents

await agent.run(ctx)

from livekit import agents

await agent.run(ctx)

The key point is that the avatar belongs in the agent process, not in the browser. The agent already owns the conversation state and audio generation; the avatar should subscribe to that same state so it can lip-sync with the correct timing. If you try to drive the face directly from the browser transcript, you will usually introduce additional delay and jitter.

Next.js frontend: join the room and render the media

On the client, the job is usually simpler than people expect. The page should authenticate the user, fetch a LiveKit token from your backend, join the room, and render the published tracks. The avatar face is just the video track published by the agent.

A minimal Next.js component might look like this:

useEffect(() => {

}, []);

useEffect(() => {

}, []);

useEffect(() => {

}, []);

Two implementation details matter here:

• Use a dedicated audio element or audio sink for the agent’s voice. Browsers can be picky about autoplay, so you may need a user gesture before playback starts.

• Keep the video element stable. Recreating the element on every state update can interrupt playback and make the face appear to stutter.

If your app is built with React Server Components, keep the LiveKit connection logic in a client component. Session tokens and room joins are runtime concerns and should not be mixed into server-rendered UI.

Session creation and API-side control

In a real support app, you usually want backend control over avatar creation and session lifecycle. That lets you bind sessions to authenticated users, enforce rate limits, and attach metadata like tenant IDs or conversation IDs.

The REST API is useful when you need to create or manage avatars and realtime sessions programmatically. A simple request shape looks like this:

curl -X POST https://api.protoface.com/v1/sessions \
}'

curl -X POST https://api.protoface.com/v1/sessions \
}'

curl -X POST https://api.protoface.com/v1/sessions \
}'

Use this pattern when your app needs to provision sessions on demand, or when you want to precompute a session before the user enters the call. The docs at docs.protoface.com are the source of truth for request fields and lifecycle semantics.

What to watch out for in production

Most problems in these integrations are not “avatar” problems; they are realtime media problems. A few practical issues show up repeatedly:

• Startup latency: if the backend waits too long to create the avatar session, the user experiences dead air before the face appears. Pre-create when possible.

• Autoplay restrictions: browsers may block audio until the user interacts with the page. Plan for a click-to-start flow.

• Layout stability: reserve space for the avatar video to avoid CLS and to keep the video element from being torn down.

• State drift: the agent, transcript, and avatar should share the same session boundary. Don’t let the frontend improvise its own conversation state.

• Quality trade-offs: higher-quality avatar output generally costs more and may add compute or rendering overhead, so choose a tier that matches the UX you actually need.

One more subtle point: if you are streaming both text and audio, the text transcript should be treated as informational, not authoritative for animation timing. The avatar should track the audio generation path, because text often arrives in bursts that do not match the final acoustic timing.

Where Protoface fits

This is the part that makes the integration much less bespoke. Protoface provides the avatar layer that can be attached to your realtime agent, so you don’t have to build face rendering, lip sync, or session management from scratch. The LiveKit plugin is the direct integration point for a voice agent, and the quickstarts in the GitHub org are the fastest way to see a working end-to-end setup. For the agent-side plugin itself, the repo at github.com/protoface-ai/protoface-plugin-pipecat is useful if you are working in a Pipecat-based stack rather than a raw LiveKit agent.

If you prefer to inspect or automate avatar/session lifecycle from Python, the Python SDK is the cleaner route than hand-rolling HTTP. And if you want to validate the user-facing experience first, the docs at docs.protoface.com cover the supported surfaces and the exact request/response shapes.

Conclusion

Adding a talking customer support avatar to a Next.js app is mostly about getting the boundaries right: keep LiveKit responsible for realtime transport, keep the agent responsible for conversation state, and keep the avatar attached to the same audio stream that drives the response. That gives you synchronized speech and video without forcing the browser to manage media orchestration.

If you want to implement this yourself, start with a small proof of concept: one backend agent, one LiveKit room, one avatar session, and a Next.js client that just joins and renders tracks. Once that works, layer in authentication, session provisioning, and whatever guardrails your support workflow needs.

For implementation details, refer to docs.protoface.com and the relevant quickstart in the GitHub org. Start with the simplest working path, then optimize for latency, stability, and cost once the media flow is proven.