Introduction

If you are building a SaaS product that includes voice agents, support bots, sales assistants, or interactive tutorials, you eventually run into the same problem: audio alone is often not enough. Users want a face. They want lip sync, a speaking state they can trust, and a visual cue that the system is live. The integration challenge is not the avatar itself; it is wiring it into an existing backend without turning your architecture into a tangle of WebRTC glue, session state, and frontend secrets.

This post shows a practical pattern for using Django as the application control plane and FastAPI as the realtime edge for avatar/session APIs. By the end, you should have a clear way to separate business logic from realtime delivery, keep API keys off the browser, and cleanly attach a talking avatar to a voice agent or customer-facing web experience.

Split the problem: Django owns product state, FastAPI owns realtime execution

The main architectural mistake I see is trying to make one framework do everything. Django is excellent for accounts, billing, permissions, admin workflows, and durable business state. FastAPI is a better fit for low-latency request/response endpoints, WebSocket or streaming-adjacent orchestration, and thin integration layers around external realtime services.

A clean pattern is:

• Django stores tenants, plans, avatar settings, agent configs, and usage policy.

• FastAPI exposes endpoints that create ephemeral sessions, proxy calls to the avatar provider, and return just enough data for the frontend or agent runtime to connect.

• The browser never sees long-lived secrets. It gets either a signed embed URL, a short-lived session token, or a backend-generated configuration payload.

This separation matters because realtime avatar sessions are operationally closer to a media pipeline than to a standard CRUD API. Session creation needs to be fast, deterministic, and isolated from slow admin tasks or ORM-heavy code paths. Django can still orchestrate the workflow, but the edge should remain thin.

Designing the control plane in Django

Start by modeling the product state that your application actually owns. For most SaaS products, that means a tenant, a user, an avatar configuration, and a session record. The exact schema depends on your product, but the important part is to avoid putting transient realtime data into your core tables.

A useful mental model is:

• Persistent objects: orgs, users, avatar presets, system prompts, brand voice settings, rate limits, plan tier.

• Ephemeral objects: a live session, a generated access token, a runtime voice instruction, a one-time embed authorization.

When a user clicks “Start avatar,” Django should validate permissions, select the right avatar config, and hand off to the realtime layer. It should not itself manage media tracks or maintain a long-lived connection.

For example, your Django view might just enqueue or call a FastAPI endpoint with the minimum set of inputs:

# Django view logic (illustrative)

return JsonResponse(resp.json())

# Django view logic (illustrative)

return JsonResponse(resp.json())

# Django view logic (illustrative)

return JsonResponse(resp.json())

The details are intentionally boring. That is the goal. Your SaaS backend should remain the source of truth for authorization and configuration, while the realtime path stays stateless enough to scale and debug.

FastAPI as the thin realtime edge

FastAPI is a good fit for the session layer because the surface area is usually small: create a session, fetch a session status, issue a one-time embed, or connect a voice-agent runtime to an avatar provider. You want short code paths, explicit schemas, and predictable response times.

Keep these rules in mind:

1. Authenticate every internal request from Django to FastAPI. Use a service token or mTLS, not user cookies.

2. Store provider API keys only on the server side. The browser should never receive them.

3. Make session creation idempotent where possible. Retrying should not produce duplicate live sessions unless that is intended.

4. Return only the data needed for the next hop: a session ID, an embed URL, or agent connection parameters.

A minimal FastAPI endpoint might look like this:

from fastapi import FastAPI, HTTPException

raise HTTPException(status_code=502, detail="session creation failed") from exc

from fastapi import FastAPI, HTTPException

raise HTTPException(status_code=502, detail="session creation failed") from exc

from fastapi import FastAPI, HTTPException

raise HTTPException(status_code=502, detail="session creation failed") from exc

In practice, you would replace the placeholder with calls through the provider SDK or REST API, but the shape stays the same: a narrow endpoint that creates or manages a realtime avatar session and returns a connection artifact.

Where WebRTC and streaming semantics matter

Realtime avatars are not just animated images. They are media sessions that usually combine audio input, synthesized audio output, and a video track generated from the avatar pipeline. The agent often needs to hear the user, produce text or speech, and then synchronize mouth movement with the emitted audio. That means timing is the whole game.

Two implementation details trip teams up:

• Transport is stateful even if your API is not. The HTTP request that creates a session is easy; the actual live interaction happens over a session that has its own lifecycle, connectivity state, and failure modes.

• Audio and video must stay synchronized. If the audio backend, model latency, or network buffering drifts, the face will look wrong even when the response text is correct.

That is why you should keep your own backend focused on orchestration and observability. Record when a session was created, what avatar and prompt were used, what quality tier was chosen, and when the session closed. Do not try to reconstruct the media pipeline from logs after the fact; instead, surface the session ID in your application telemetry so support can correlate user issues with provider-side events.

For voice-agent products, the best integration point is usually inside the agent runtime itself, not in your web app shell. Your web backend can create the session, but the agent process needs to own the live media connection and feed the output into the avatar layer.

Attaching a face to a voice agent with the LiveKit plugin

If your stack already uses LiveKit Agents, the simplest route is to insert the avatar as a plugin in the agent runtime rather than building a custom media bridge. That keeps your application code focused on business logic while the agent handles turn-taking, speech synthesis, and conversation state.

The plugin pattern is straightforward: the agent continues to process audio and text as usual, and the avatar plugin mirrors the agent’s speaking state into a synchronized video face. The integration is especially useful for support bots and call-center style experiences where the user already expects a conversational loop.

A small example, with the details intentionally abbreviated, looks like this:

# LiveKit Agents example (illustrative)

)

# LiveKit Agents example (illustrative)

)

# LiveKit Agents example (illustrative)

)

The key point is not the exact constructor signature, which you should verify in the docs, but the integration shape: the voice agent owns conversation flow, and the avatar plugin turns that flow into a lip-synced face. If you want a concrete starting point, the plugin repo is the right place to look: GitHub examples for the plugin.

When the browser should embed, and when it should not

For some products, you do not want any custom frontend work at all. You just want to drop an interactive avatar into a marketing site, help center, or product landing page. In those cases, an iframe-based embed is the cleanest boundary: the browser loads the avatar UI from the provider, while your site only controls the container and allowed origin.

This model is safer than exposing keys or trying to run the session protocol in JavaScript. The parent page can pass only per-embed settings, such as voice or instructions, and the embed layer can enforce origin allowlists, per-IP and duration limits, and other controls on the server side. For most SaaS teams, that is exactly the right trade-off: less flexibility than a fully custom client, but much better security and much lower implementation cost.

A simple embed flow on your side often boils down to a backend-generated URL that your page renders in an iframe. The browser never sees the API key, and your application still remains the authority for who can start which avatar and under what limits.

Using the REST API or Python SDK for backend orchestration

For direct integration work, the REST API is useful when you want to create avatars, manage sessions, or inspect usage from your server. A Python backend can call it directly or use the Python SDK when you want a cleaner object model. The choice is mostly about ergonomics; the security model is the same either way: keep credentials server-side.

A minimal cURL request might look like this:

curl -X POST https://api.protoface.com/sessions \
}'

curl -X POST https://api.protoface.com/sessions \
}'

curl -X POST https://api.protoface.com/sessions \
}'

If you prefer Python, the SDK gives you a more maintainable path for service code. The exact method names and request fields are documented, but the usage pattern is generally the same: instantiate the client with your server-side key, create or update the resource, then persist the returned session metadata in your own database.

# Python SDK usage (illustrative)

print(session.id)

# Python SDK usage (illustrative)

print(session.id)

# Python SDK usage (illustrative)

print(session.id)

If you want the canonical API shape before wiring this into your app, check the public docs at docs.protoface.com.

Operational gotchas that matter in production

A few things are worth handling explicitly from day one:

• Timeouts and retries: session creation should fail fast, but retries must be guarded so you do not accidentally create duplicate live sessions.

• Rate limiting: tie limits to tenant, IP, and plan tier. Realtime systems can be abused quickly if you leave them open-ended.

• State cleanup: end stale sessions and mark them terminal in your database, even if the media layer times out or disconnects unexpectedly.

• Observability: log correlation IDs from Django through FastAPI and into the provider call. Support will need this later.

• Prompt/config versioning: store the exact avatar instructions or agent prompt version that created the session so behavior can be reproduced.

If you build the integration this way, the avatar layer becomes another managed capability in your SaaS stack rather than a special-case subsystem. That is the real win.

Conclusion

The practical pattern is simple: use Django for durable product state and authorization, use FastAPI for thin realtime orchestration, and keep avatar/session credentials off the client. For LiveKit-based voice agents, attach the avatar at the agent layer; for web embeds, let the iframe own the realtime boundary. In both cases, your backend should create and govern sessions, not simulate the media stack itself.

If you are implementing this now, start with the docs, pick the integration surface that matches your product, and keep the first version narrow. The fewer moving parts you expose to the browser, the easier it is to ship something reliable.

For API details, SDK usage, and current examples, see the documentation and the relevant quickstarts in the public repos.