Introduction

Adding a realtime avatar to a Next.js app sounds simple until you actually wire it up: you need low-latency transport, a streaming audio pipeline, synchronized lip motion, stateful session control, and a UI that doesn’t become a maintenance problem the first time you switch providers or change deployment targets.

This post shows a practical way to build an accessible avatar surface in Next.js and TypeScript without turning your frontend into a media stack. By the end, you should understand the moving pieces of a realtime avatar UI, how to separate browser concerns from backend session management, and how to keep the experience usable for keyboard users, screen readers, and degraded network conditions.

We’ll use Protoface as the backend avatar service and focus on the integration pattern rather than any one demo implementation.

What “realtime avatar” actually means in practice

A realtime avatar interface is not just “play a video of a face.” It is usually a bidirectional session where:

• Audio is streamed from a voice agent or client into a realtime session.

• The avatar’s video is generated incrementally and stays synchronized with the audio.

• Session state survives reconnects, interruption, and turn transitions.

• The frontend can reflect connection, speaking, listening, and error states clearly.

From an application standpoint, you are generally dealing with two separate planes:

1. Control plane: create avatars, start sessions, configure voice/instructions, and retrieve session metadata.

2. Media plane: receive and render the realtime stream, usually through WebRTC or an iframe that encapsulates it.

This separation matters. The control plane belongs on a trusted backend. The media plane belongs in the browser, but only if you can keep secrets out of the client and keep the rendering path accessible.

Building the Next.js shell correctly

In Next.js, the main architectural choice is to keep avatar session creation server-side and render only an opaque client component in the browser. That gives you a few benefits:

• You never expose API keys in the client bundle.

• You can validate user identity, rate-limit session creation, and attach per-session metadata.

• You can choose the rendering surface later without rewriting the app.

A simple pattern is:

1. Create a server route or server action that requests a session from your backend.

2. Return only the minimum browser-safe payload needed to attach the avatar surface.

3. Render a client component that handles loading, focus management, and announcements.

Server-side session creation

Below is an illustrative Node route handler. The exact fields depend on your avatar/session model, so treat this as a shape, not a contract:

import { NextResponse } from 'next/server';

}

import { NextResponse } from 'next/server';

}

import { NextResponse } from 'next/server';

}

Two implementation details matter here:

• Use environment variables only on the server. Do not forward the API key to the browser.

• Keep the payload minimal. The frontend should not need more than a session identifier, token, or embed URL, depending on your chosen integration surface.

Accessible video and interaction patterns

Accessibility is where many realtime media UIs fail. An avatar is visually rich, but the actual product behavior is conversational. That means the important information is often the dialogue state, not the animation itself.

Good defaults:

• Announce state changes: “Connecting,” “Listening,” “Speaking,” “Reconnecting,” and “Error” should be visible and exposed to assistive tech.

• Use proper landmarks: wrap the avatar and controls in a labeled region or dialog if the interaction is modal.

• Keep controls keyboard reachable: mute, retry, stop session, and transcript toggles should all be buttons with clear focus states.

• Do not rely on video alone: provide transcript text or at least a short status summary adjacent to the player.

For screen readers, the avatar element itself is usually decorative unless the visual pose contains semantically important data. In most conversational apps, mark the video surface as presentation-only and put the meaning in text:

<div aria-labelledby="avatar-title">
</div>

<div aria-labelledby="avatar-title">
</div>

<div aria-labelledby="avatar-title">
</div>

If you render a real video element, keep it responsive and avoid forcing it to autoplay with audio. Browsers are strict about autoplay policies, and users should be able to start playback intentionally. If your session is full-duplex, make sure the audio path is explicit: microphone permission, playback permission, and fallback messaging when either fails.

Frontend state management: treat the avatar as a session, not a widget

A realtime avatar component tends to accumulate state quickly:

• connection lifecycle

• mic permission status

• active speaker / turn state

• transcript history

• network retry logic

In React, model these states explicitly. A compact reducer or state machine usually ages better than a handful of booleans.

type AvatarState =
| { status: 'error'; message: string };

type AvatarState =
| { status: 'error'; message: string };

type AvatarState =
| { status: 'error'; message: string };

This is useful for accessibility too, because each state can map to one visible label and one live-region message. That prevents contradictory UI such as “Connecting” and “Speaking” appearing at the same time.

Handling failures and network edges

Realtime media sessions fail in predictable ways: permission denied, connection timeout, transient network loss, and server-side session expiration. Build for those explicitly.

A few practical rules:

• Retry only idempotent setup. It is reasonable to retry session bootstrap, but not to blindly recreate a user-visible session after the user has already spoken.

• Surface reconnecting state. If the transport drops, tell the user what is happening instead of freezing the interface.

• Separate transcript from transport. If video stalls, the transcript and controls should still be usable.

• Instrument the lifecycle. Log session creation time, media attach time, and disconnect reason so you can debug real user sessions.

Also pay attention to layout stability. A loading avatar that shifts size when the stream arrives is a common accessibility bug because it causes focus and reading order problems. Reserve the final aspect ratio up front.

Where Protoface fits naturally

The cleanest integration point for a Next.js app is usually the REST API for server-side session creation, or an iframe embed if you want to avoid handling media plumbing in your app entirely. For developer teams that want to keep their frontend thin, the iframe model is attractive: the browser gets an embedded interactive avatar, while the API key stays server-side or never reaches the browser at all. The embed controls also let you enforce parent-origin allowlists and simple per-embed constraints without inventing your own session gate.

If you need to create or inspect sessions programmatically, the API and Python SDK are the right tools; if you are wiring a voice agent, the LiveKit plugin is the relevant surface. The key point is that the avatar layer is a backend capability, not a UI gimmick. That keeps your Next.js code focused on presentation, accessibility, and app logic.

Python and LiveKit examples for the backend path

For backend automation, the Python SDK is a good fit when you want to provision avatars or sessions from jobs, admin tools, or agent orchestration code. The exact method names are documented, but the pattern looks like this:

from protoface import Client

print(session)

from protoface import Client

print(session)

from protoface import Client

print(session)

If you are using LiveKit Agents, the plugin path is more direct because the avatar becomes part of the voice agent pipeline. Conceptually, the agent still handles the conversation, while the plugin adds a synchronized face to the stream. That is the right abstraction when your system is already built around agent turns and you want the avatar to track them automatically.

# illustrative only; check the repository for the current API

)

# illustrative only; check the repository for the current API

)

# illustrative only; check the repository for the current API

)

If you are starting from scratch, the quickest way to validate the agent-side integration is the plugin repo and its examples: https://github.com/protoface-ai/protoface-plugin-pipecat. For general API reference, use the docs at https://docs.protoface.com.

Practical Next.js implementation checklist

When you put this together in a production app, I would recommend the following order:

1. Implement a server route that creates a session using your API key.

2. Render a client avatar component that starts in a clearly labeled idle state.

3. Add keyboard-accessible controls and a visible status region.

4. Test with autoplay disabled, mic permission denied, and a throttled network.

5. Verify that no secret appears in the browser bundle or request logs.

If you are also building the voice agent side, add a transcript and logging path before polishing the animation. In practice, users will forgive modest visuals if the conversation is fast, legible, and recoverable. They will not forgive an avatar that looks polished but fails silently when the connection drops.

Conclusion

The core pattern is straightforward: keep avatar/session control on the server, keep the browser focused on rendering and interaction, and design the UI around conversational state instead of video frames. That gives you a cleaner security model, fewer accessibility regressions, and a frontend that is much easier to maintain in Next.js and TypeScript.

If you want to go deeper, start with the docs at https://docs.protoface.com, then choose the integration surface that matches your stack: REST API for custom app flows, the Python SDK for automation, the LiveKit plugin for voice agents, or an iframe embed when you want to ship quickly without exposing backend secrets.