Introduction

If you want an AI avatar to feel responsive in a web app, the hard part is not rendering a video element. The hard part is keeping audio, speech synthesis, lip sync, and browser transport aligned closely enough that the avatar looks like it is actually participating in the conversation.

This post shows how to embed an ElevenLabs-powered avatar in a Next.js app using WebRTC transport. By the end, you should understand the transport and rendering pieces, how to wire them into a React component, and where the common latency and reliability traps are.

For context, Protoface is the layer that provides the avatar/session plumbing here: you create or manage an avatar session, then connect it into your app as a realtime media stream. The concepts are the same whether you are building a customer-support bot, a sales assistant, or a conversational demo embedded on a landing page.

What WebRTC is doing in this architecture

WebRTC is the right transport when you need low-latency media between a browser and a realtime service. In this case, the browser is not just “playing a video”; it is participating in a peer-style media session that can carry the avatar video track and, depending on your setup, audio in the same realtime pipeline.

The practical benefit is jitter-tolerant, adaptive delivery. Compared with polling or progressive video delivery, WebRTC is much better suited to a face that must react quickly to user speech. You still need to budget for model latency and network latency, but the transport itself is optimized for interactive media.

A useful mental model:

• The LLM decides what to say.

• The TTS layer generates the audio, in this case from ElevenLabs.

• The avatar service renders a lip-synced face from that audio.

• WebRTC carries the media to the browser with low end-to-end latency.

The important thing is that lip sync is driven by the timing of the generated speech, not by the browser “guessing” where mouth shapes should go. If the transport or buffering gets too large, the face still works, but the interaction stops feeling realtime.

Next.js client setup: keep media code on the client

In Next.js, the WebRTC connection and any DOM/media work must live in a client component. If you use the App Router, start with "use client" and keep all session setup inside effects or event handlers.

The exact session fields depend on your avatar/session API, but the pattern is stable: create a session on your backend, return the ephemeral connection data to the browser, and let the browser attach the remote tracks.

"use client";

}

"use client";

}

"use client";

}

There are two implementation details worth calling out:

• playsInline matters on mobile browsers; without it, playback behavior can be inconsistent.

• Keep the local element muted unless you explicitly want the browser to render audio locally. In most avatar apps, the remote audio track is what matters, and you avoid feedback by controlling output intentionally.

Also, don’t create the peer connection during render. Re-renders happen for reasons unrelated to media state, and reconnecting on every render is a great way to break your session.

Session creation: keep API keys on the server

The browser should never see your long-lived API key. If you need to create sessions dynamically, do it from a server route in Next.js, then return only the session-specific values the client needs.

Here is the basic shape using the REST API directly from your server. The exact payload depends on the avatar/session configuration you are using, but the authorization pattern is the important part:

curl -X POST https://api.protoface.com/v1/sessions \
}'

curl -X POST https://api.protoface.com/v1/sessions \
}'

curl -X POST https://api.protoface.com/v1/sessions \
}'

In a Next.js route handler, you would proxy that call and return the session material to the browser. The browser then completes the WebRTC handshake. This split is the same pattern you should use for any secret-bearing realtime integration: the server owns credentials; the client owns ephemeral session state.

If you prefer to keep the session lifecycle behind typed client code, the Python SDK is useful for backend orchestration, scheduled jobs, or agent-side provisioning. For example:

from protoface import Client

print(session.webrtc)

from protoface import Client

print(session.webrtc)

from protoface import Client

print(session.webrtc)

Again, treat the field names as illustrative unless you are copying from the docs. The point is that session creation belongs on the trusted side of your app, not in a browser bundle.

ElevenLabs, lip sync, and latency budgeting

When ElevenLabs is the speech source, your real bottleneck is usually not WebRTC itself; it is the combined time for model response, speech synthesis, avatar rendering, and browser decode. If you want the experience to feel conversational, you need to reason about that full path.

A few practical rules:

• Prefer streaming generation over waiting for a full utterance. The avatar can start moving sooner if audio arrives incrementally.

• Keep your turn-taking logic strict. If the user interrupts, cancel the current synthesis and stop the avatar from “finishing” a stale response.

• Use a stable network path for development when measuring latency. Wi-Fi variance can hide transport regressions.

• Separate render latency from model latency. A smooth-looking face that arrives 1.5 seconds late is still a bad UX.

The browser side should also handle media failures gracefully. A dropped peer connection, autoplay restriction, or tab backgrounding event should not take down the whole UI. Show a retry state, keep the transcript visible, and reconnect explicitly.

Using Protoface without leaking secrets into the browser

This is the part that tends to matter most in production. If you are embedding a realtime avatar directly in a customer-facing Next.js app, the safest pattern is:

1. Your server creates or fetches a session using the REST API.

2. The client receives only ephemeral signaling data.

3. The browser establishes a WebRTC connection and renders the remote video track.

That means your API key stays on the server, while the user’s browser only participates in the session it was explicitly given. If you are building a “voice agent with a face,” this is also where you would wire the avatar session to your existing agent orchestration.

For teams already on LiveKit, the ElevenLabs agents quickstart is the closest reference point for how the media side fits together in a realtime agent stack. The useful pattern is the same even if your app is not using LiveKit end-to-end: keep the avatar as a media participant, and keep session control server-side.

If you are already exploring the public docs, the documentation is where you should verify the exact session schema, transport options, and any avatar configuration fields before you wire this into production code.

Operational gotchas in a Next.js deployment

There are a few things that usually bite teams the first time they ship this.

1. Serverless timeouts. If session creation or signaling happens in a serverless route, make sure your platform timeout comfortably exceeds any upstream calls. Session setup should be quick, but cross-region API hops add up.

2. React Strict Mode. In development, effects may run twice. If you connect on mount, guard against duplicate peer connections or duplicate session creation.

3. Autoplay policies. Even with a remote media track, browsers can still block playback until the page has received a user gesture. If your avatar must speak immediately, design the UI around a click-to-start flow.

4. Cleanup. Close the peer connection on unmount and dispose of any media tracks. Realtime media leaks do not usually show up as obvious errors; they show up as flaky reconnects and rising resource use.

5. Observability. Log signaling timestamps, connection state changes, and utterance start times. If something feels “slow,” you want to know whether the delay is in the model, TTS, rendering, or transport.

Conclusion

Embedding an ElevenLabs-backed avatar in Next.js is mostly an exercise in treating the avatar as a realtime media participant, not as a static video asset. WebRTC handles the low-latency transport; your server handles secrets and session creation; the browser handles rendering and connection state.

If you keep those boundaries clean, the implementation stays manageable: server-side session creation, client-side peer connection, and explicit handling for autoplay, reconnects, and interruptibility.

For concrete request shapes, SDK calls, and current transport details, start with docs.protoface.com. If you want an implementation reference, the quickstarts linked from the repo are a good way to validate the full path before you adapt it to your own app.