Introduction

If you want a support agent that can talk, listen, and keep a synced video face while it answers customers, the hard part is not “making video.” The hard part is keeping the avatar tenant-safe, low-latency, and operationally sane when multiple customers, environments, and agents are involved.

This post walks through the design of a tenant-safe realtime avatar support agent with FastAPI on the backend and Next.js on the frontend. By the end, you should have a clear pattern for:

• issuing per-tenant avatar/session credentials from your backend,

• streaming avatar media into a browser client without exposing API keys,

• binding session state to a tenant and a support ticket, and

• handling the failure modes that show up in real deployments: stale sessions, user disconnects, and cross-tenant leakage.

I’ll keep the code short and pragmatic. Exact request fields depend on the API version, so treat the examples as structural patterns and verify the payloads in the docs.

What “tenant-safe” actually means

In a SaaS support workflow, “tenant-safe” is more than just putting a tenant ID in a table. It means every realtime avatar session is created, used, and torn down within a tenant boundary. Concretely:

• One tenant cannot reuse another tenant’s avatar/session token.

• Browser clients never see a long-lived platform API key.

• Per-session instructions, voice selection, and conversation context are scoped to the current customer request.

• Rate limits, origin checks, and session duration limits are enforced before media starts flowing.

This matters because realtime media paths are stateful. Once a browser connects over WebRTC or a similar streaming transport, you are no longer just serving JSON. You are managing identity, authorization, and lifecycle on a live connection.

Backend pattern: create sessions in FastAPI, not in the browser

The safest architecture is to keep your platform credentials on the server and expose only short-lived session material to the frontend. In practice, a browser should ask your FastAPI backend for a tenant-scoped session token or embed URL, and your backend should call the avatar service on its behalf.

A minimal FastAPI endpoint usually does three things:

1. authenticates the requesting user against your own app,

2. checks that the user belongs to the tenant they claim, and

3. creates a new avatar session with tenant-specific settings.

from fastapi import FastAPI, Depends, HTTPException

from fastapi import FastAPI, Depends, HTTPException

from fastapi import FastAPI, Depends, HTTPException

The exact endpoint and fields may differ, but the architectural rule does not: your backend is the policy boundary. The browser gets a short-lived session artifact, not the master key.

Browser pattern: Next.js consumes a session, then renders the avatar

On the frontend, your Next.js app should stay dumb: fetch session data from your backend, then hand that data to the avatar player or iframe. This keeps tenant logic centralized and avoids duplicating authorization rules in JavaScript.

If you are embedding a customer-managed iframe, the browser never sees a platform API key at all. That is the cleanest path when you want a support widget on a public site with tenant isolation, parent-origin allowlisting, and per-embed limits handled by the platform.

import { useEffect, useState } from "react";<p></p>

import { useEffect, useState } from "react";<p></p>

import { useEffect, useState } from "react";<p></p>

The important part is not the iframe tag itself; it is the boundary around it. Your backend should mint a session that is valid for one tenant, one support interaction, and a constrained window of time. That gives you a simple revocation story: expire the session and it is done.

Managing realtime state without making a mess

Realtime avatar systems break in boring ways if you treat them like stateless HTTP. You need to design around session lifecycle.

There are a few practical rules that keep things stable:

• Bind session state to a ticket or conversation ID. If the user refreshes, you need to know whether to resume, recreate, or discard.

• Store tenant ID server-side. Do not trust a tenant ID sent from the browser without checking it against the authenticated user.

• Use short-lived session artifacts. Realtime sessions should not outlive the interaction by much.

• Make disconnects idempotent. A user can close the tab, lose network, or retry a session start request. Cleanup logic should tolerate duplicates.

In a support setting, I usually keep three states in the app database: pending, active, and closed. The backend can translate those into avatar session creation and teardown calls, while the UI only reflects the current state.

from enum import Enum<p></p>

from enum import Enum<p></p>

from enum import Enum<p></p>

Voice, instructions, and support behavior

The avatar is only useful if it behaves like a support agent rather than a generic chatbot in a face. That behavior should be configured per session, not baked into the client.

Typical per-session controls include:

• Instructions: “Answer briefly, ask one question at a time, never invent account details.”

• Voice choice: pick a voice that matches the tenant’s brand, or keep it static for consistency.

• Context payload: pass ticket metadata, language, plan tier, or order status from your backend.

This is where tenant safety and prompt safety overlap. If the wrong customer context leaks into a session, the agent can literally speak it on video. Keep context construction on the server and sanitize it like any other privileged data flow.

Where the LiveKit plugin fits

If your agent already runs inside a LiveKit voice pipeline, the cleanest way to give it a synchronized talking face is the LiveKit Agents plugin. That path lets you keep your speech stack where it already is and add the avatar as a media surface, rather than rewriting your agent around a new runtime.

The integration lives in the plugin package on PyPI, and the examples in the plugin repo are the right place to start if your architecture is agent-first rather than web-embed-first: plugin examples and the Pipecat integration guide at docs.pipecat.ai.

A sketch looks like this:

# Illustrative only; exact setup depends on your agent stack

# Illustrative only; exact setup depends on your agent stack

# Illustrative only; exact setup depends on your agent stack

The value here is synchronization. If your voice agent is already producing text-to-speech and turn-taking decisions, the avatar layer should consume the same turn events so the mouth movement, gaze, and speech stay coherent.

Operational gotchas: latency, retries, and abuse controls

Once this is live, most issues are operational rather than algorithmic.

Latency. Realtime avatars amplify any delay in your speech pipeline. Keep the backend session creation path fast, avoid blocking calls in the request path, and cache only what is safe to cache. If the agent has to wait on your database or ticketing system, do that before session start when possible.

Retries. If the frontend retries “create session” after a timeout, make sure your backend either deduplicates by tenant/ticket or can safely create a second session without violating policy. Idempotency keys help here.

Abuse controls. Public-facing embeds need origin allowlists, per-IP limits, and duration limits. Those are not nice-to-haves; they are the difference between a controlled support widget and an open relay for your API budget.

Observability. Log tenant ID, ticket ID, session ID, and lifecycle transitions. Do not log raw secrets or customer audio/text unless you have a concrete retention policy.

How Protoface fits this architecture

The platform piece that matters most here is the customer-managed iframe embed and the backend API. The iframe keeps secrets off the client entirely, while the REST API lets your FastAPI service create tenant-scoped sessions and manage lifecycle from a trusted backend. If you are wiring the avatar into an existing voice-agent stack, the plugin route is better; if you are embedding a support surface into a web app, the iframe route is simpler and safer. The docs at docs.protoface.com cover the exact session fields, auth, and embedding flow.

Conclusion

A tenant-safe realtime avatar support agent is mostly an exercise in boundary design: keep credentials server-side, mint short-lived session artifacts, bind every session to a tenant and ticket, and treat realtime media like a privileged stateful connection instead of a regular API response.

For a FastAPI + Next.js stack, the practical pattern is straightforward: backend creates and authorizes the session, frontend renders the session artifact, and the avatar layer consumes tenant-scoped instructions and context. If you want to extend an existing voice agent, the LiveKit plugin path is the right fit; if you want an embeddable support surface with no browser-exposed API key, use the iframe model.

Start with the docs, then wire up one tenant end-to-end before generalizing to multi-tenant routing and billing.