Introduction

If you already have a voice agent, adding a realtime video face is mostly an integration problem: you need a low-latency media path for the avatar stream, a synchronization point for speech and visemes, and a safe way to configure and operate the session. The hard part is not “making a talking head”; it’s making one that stays aligned with the agent’s audio, doesn’t leak credentials, and can be embedded or orchestrated in a sane way.

This post shows the practical shape of that integration in a Next.js app. By the end, you should understand how a realtime avatar session fits into a web app, how simulcast affects delivery quality and bandwidth, what to watch for in browser/WebRTC integration, and where Protoface fits when you want to get from “working demo” to something you can ship.

What “realtime avatar” means in practice

A realtime AI avatar is usually a streaming video producer attached to an agent that is already producing or receiving speech. The avatar is not a pre-rendered clip. It has to respond continuously to new audio, which means the transport matters: the browser should receive a live media track with low latency, and the agent side should be able to send text, audio, or control events often enough to keep mouth motion and expression in sync.

In a Next.js app, you typically split responsibility into three pieces:

• Browser UI: starts and renders the session, subscribes to the avatar video, handles lifecycle.

• Backend/API: creates or authorizes realtime sessions and returns the minimal token/metadata needed by the browser.

• Agent runtime: generates speech and pushes the avatar state in lockstep with the conversation.

The key principle is: keep secret material off the client, and keep media transport separate from business logic. That means the browser should never see your long-lived API key, and your avatar session should be treated like any other realtime media session: ephemeral, scoped, and revocable.

Simulcast: why you want it for avatars

Simulcast means publishing multiple encodings of the same video track at different bitrates/resolutions. On the receiving side, the network or SFU can choose an appropriate layer based on available bandwidth and client conditions. For a talking avatar, this matters more than it might sound:

• On fast connections, users get a crisp face with fewer compression artifacts.

• On weaker connections, playback can downshift instead of stalling the whole experience.

• During UI resizing or background tabs, you avoid wasting bandwidth on a high layer the user can’t really see.

The trade-off is a little more complexity in publishing and sometimes a bit more CPU on the sender. In exchange, you get a much more robust experience across devices and network conditions. For a realtime avatar, that is usually the right default.

In a Next.js app, you usually don’t manage the encodings directly unless you’re building the media pipeline yourself. Instead, your job is to ensure that the session you join is configured for a video publisher that supports simulcast, and that your client subscribes cleanly to the track without forcing unnecessary transcoding or canvas work in the browser.

Next.js integration pattern

The cleanest pattern is to keep the client thin. In practice, that looks like:

1. Create a server route in Next.js that requests or prepares a realtime session.

2. Return only ephemeral connection details to the browser.

3. Render a component that joins the session and attaches the avatar track to a DOM element.

What you should not do is put a permanent API key in a client component, localStorage, or a public env var. That is the most common mistake when teams first wire up realtime media services.

A minimal browser-side component usually ends up looking like a regular WebRTC or LiveKit-style join flow: fetch session info, connect, subscribe to the video track, then mount the track into a container. Exact APIs vary by provider, but the shape is stable.

import { useEffect, useRef } from "react";

}

import { useEffect, useRef } from "react";

}

import { useEffect, useRef } from "react";

}

The important part is not the exact client library. It’s the separation of concerns: your app acquires an ephemeral session on the server, and the browser only handles media playback.

Backend: create sessions, do not expose secrets

If you need to create sessions programmatically, use the REST API from server-side code or your deployment backend. That keeps the authorization header where it belongs and lets you apply your own app logic before a session is issued.

curl -X POST https://api.protoface.com/v1/sessions \
}'

curl -X POST https://api.protoface.com/v1/sessions \
}'

curl -X POST https://api.protoface.com/v1/sessions \
}'

The exact request fields depend on the endpoint and session type; use the docs for the authoritative schema. The pattern remains the same: authenticated server-to-server creation, then a short-lived client handoff.

For a Next.js app, that usually means one route handler that creates the session and returns only the fields the browser needs to join. If you want to add rate limits, usage accounting, or customer-specific behavior, do it before the session is issued. That keeps your media layer stateless and easier to debug.

Where Protoface fits

Protoface is useful when you want the avatar layer without building and maintaining the whole video face stack yourself. For a Next.js integration, the most relevant surface is the developer-facing session API plus the existing quickstart patterns documented in the docs. The session model lets you keep API keys server-side, create a realtime avatar session, and hand the browser only the minimal connection data needed to render the stream.

That is also where simulcast matters operationally: you are not trying to ship one fixed-quality stream to every viewer. You want a session setup that can adapt to actual client conditions, especially if the avatar is part of a customer-support flow or sales conversation where dropped frames and long buffer times are noticeable.

If you prefer to work from a reference implementation, the quickstart repositories are a good way to inspect the integration shape before wiring it into your own app. For developers already using a voice agent stack, the LiveKit path is particularly relevant; the Pipecat integration and the Pipecat service docs are useful if your agent runtime is already built around that framework.

Common implementation gotchas

A few issues show up repeatedly in avatar integrations:

• Trying to render the stream like a normal image: use the media track, not a static poll loop.

• Letting the browser handle long-lived credentials: use server-side session creation and short-lived join data.

• Ignoring audio/video sync: the avatar must follow the agent’s speech timing, not just receive text after the fact.

• Forcing full quality everywhere: simulcast exists so clients on slower networks can still participate.

• Not cleaning up tracks on route changes: in Next.js, disconnect on unmount or navigation to avoid dangling media sessions.

Also be realistic about latency budgets. If your voice agent already takes noticeable time to think, the avatar should not add another avoidable delay. Join the session early, attach the media element promptly, and keep any extra UI work off the critical path.

Conclusion

Embedding a realtime AI avatar in Next.js is mostly about getting the media architecture right: create sessions server-side, keep secrets out of the browser, subscribe to the live video track cleanly, and let simulcast handle bandwidth variability. Once those pieces are in place, the avatar becomes just another realtime surface in your app rather than a special-case integration.

If you want to go deeper, start with the public documentation at docs.protoface.com, then compare the quickstart patterns against your existing voice-agent stack. If you already have a Next.js app and a backend route, you should be able to wire up a first working avatar session without changing your app architecture much at all.