Introduction

If you are adding a talking avatar to a voice agent or interactive web app, the real design question is not “can I stream video?” It is: where should the avatar run, who owns the realtime connection, and how much application logic do you want in your browser?

There are two common integration patterns:

• Direct SDK integration inside your agent runtime or backend, where you create sessions and attach the avatar to your existing voice pipeline.

• Iframe embed, where the avatar is hosted and managed as an embedded widget with browser-safe isolation and no API key exposure.

This post walks through both approaches, the architecture trade-offs behind them, and the practical implementation details you need to avoid the usual realtime gotchas: token leakage, mismatched audio/video timing, lifecycle bugs, and awkward failure handling. By the end, you should be able to choose the right surface for your product and wire it up without guesswork.

How realtime talking avatars actually fit into an app

A “talking avatar” is usually a streaming video surface driven by a realtime speech pipeline. The important point is that the video is not independent media; it is synchronized to the agent’s speech generation and playback state. In practice, you have three moving parts:

1. Text or audio input from the user.

2. Agent reasoning and speech synthesis, which produces the response content and/or audio frames.

3. Avatar rendering, which lip-syncs the face to the spoken output and streams frames to the client.

For developer experience, the integration choice is mostly about ownership boundaries. If your app already has a backend, agent runtime, or WebRTC session manager, direct SDK/API integration usually gives you the most control. If you want to drop an avatar into a site without exposing backend credentials or building session infrastructure, an iframe embed is much simpler.

Direct SDK integration: best when the avatar is part of your agent runtime

Use a direct integration when the avatar is not just a UI widget, but a first-class part of the agent system. That is the typical case for voice agents, customer-support bots, sales reps, or in-game NPCs that already run through a backend orchestration layer.

What you gain

With direct integration, you can create and manage sessions from your server, control avatar lifecycle explicitly, and coordinate the avatar with your own agent events. That means you can tie avatar state to the same conversation state machine you already use for voice, transcripts, tool calls, and interruption handling.

This is also the cleanest option if you need per-user business logic before the avatar starts, or if you want to create sessions dynamically based on user permissions, tenant settings, or conversation context.

Python SDK example

The Python SDK is a good fit for backend services that create sessions or manage avatars programmatically. Exact request fields depend on the API version, but the shape is straightforward: authenticate with an API key, create a session, and pass the resulting session details to your client or agent runtime.

from protoface import ProtofaceClient

from protoface import ProtofaceClient

from protoface import ProtofaceClient

In a real app, you would not hardcode keys or assume those exact field names. Use the docs for the current schema and any environment-specific settings: docs.protoface.com.

REST API example

If you prefer to keep the integration language-agnostic, the REST API is enough to provision sessions from any backend. This is especially useful if your control plane is not Python, or if your agent framework already knows how to call HTTP endpoints.

curl -X POST https://api.protoface.com/sessions \

curl -X POST https://api.protoface.com/sessions \

curl -X POST https://api.protoface.com/sessions \

The important operational point is that the API key stays on the server. The browser should receive only the minimum session material needed to connect to the media stream, never your long-lived secret.

Things to get right in direct integrations

• Session lifecycle: create the avatar session after your own auth/authorization checks, and tear it down when the conversation ends.

• Audio/video timing: if your voice stack supports barge-in or interruption, make sure the avatar stops or retimes cleanly when speech is cut off.

• Reconnect behavior: WebRTC and streaming sessions will disconnect eventually. Make reconnection explicit instead of assuming the browser will “just recover.”

• State ownership: decide whether the backend, client, or agent framework owns the authoritative conversation state. Ambiguity here is where most bugs start.

LiveKit plugin integration: the cleanest path for voice agents

If your agent already runs in LiveKit, the direct path is usually to drop the avatar into the agent pipeline rather than bolt video on afterward. Protoface provides a LiveKit Agents plugin, published as livekit-plugins-protoface on PyPI, so the avatar becomes part of the same realtime agent process that handles speech and conversation flow.

That matters because the avatar should track the agent’s speaking state, not merely the existence of an audio track. When the agent is thinking, interrupted, or handing off a turn, the face should reflect that. A plugin integration keeps those transitions close to the agent logic instead of trying to reconstruct them from the browser side.

For implementation examples and the current package usage, the plugin repository is the right place to start: GitHub repo. If your stack is Pipecat-based, the integration guide is also worth reading: Pipecat Protoface guide.

Iframe embed: best when you want zero backend exposure in the browser

The iframe model is the opposite trade-off: instead of embedding your own client-side session logic, you embed a managed avatar surface. The iframe is appropriate when you want to add an interactive face to a website with minimal application code and without exposing an API key in the browser.

Why teams choose the iframe route

The big win is security and simplicity. The customer-managed embed flow lets you configure the avatar from a controlled context, then place it on a page as an <iframe>. Because the realtime avatar session is hosted behind the embed, your browser code never sees a secret key.

That makes it a good fit for marketing sites, product demo pages, or customer-facing web experiences where the avatar is important, but your frontend does not need to directly orchestrate the underlying session.

Operational constraints you should understand

The iframe model is intentionally opinionated. Expect controls like:

• Parent-origin allowlisting so only approved websites can embed the avatar.

• Per-embed voice and custom instructions so different pages can present different personalities or behaviors.

• Per-IP and duration rate limits to prevent abuse and keep usage bounded.

Those constraints are useful because they move edge-case handling out of your app code and into the embed policy. The trade-off is that you give up some low-level control. If your product needs tight coupling to your own state machine, direct integration is still the better fit.

When iframe beats direct SDK

Choose the iframe if most of these are true:

• You want the fastest path to production on a web page.

• You do not want to manage realtime session plumbing in the browser.

• You need to avoid exposing API keys entirely on the client.

• Your avatar behavior can be configured up front, rather than driven by complex application events.

Choose direct SDK integration if you need fine-grained orchestration, custom backend logic, or deeper coupling to an existing voice agent runtime.

Protoface in practice

Protoface provides both surfaces, which is useful because the right integration really depends on where your realtime control plane lives. If you are building a LiveKit-based voice agent, the plugin path is the lowest-friction way to synchronize an avatar with speech. If you are embedding on a website, the iframe flow keeps secrets server-side and gives you a controlled, production-friendly surface.

The platform also exposes the REST API and Python SDK for cases where you need to create sessions dynamically from your backend, or manage avatars and usage from code. The practical pattern is simple: keep secrets server-side, create sessions as late as possible, and let the client connect only to the specific live session it needs.

Conclusion

For realtime talking agents, the main integration decision is whether the avatar is part of your backend agent system or a managed widget in the browser.

If the avatar needs to follow your own conversation state, use direct SDK or REST integration, and for LiveKit-based agents, prefer the plugin path. If you need a fast, secure website embed with no browser-side secret handling, use the iframe model.

Either way, the key implementation rules are the same: keep API keys off the client, make session ownership explicit, and treat media timing as a first-class part of the system.

For exact request shapes, SDK methods, and quickstarts, start with the docs and the example repos linked from there. If you are integrating today, that will save you more time than guessing at the API surface.