Introduction

If you are building a voice agent with a visual face, there is an obvious temptation: call text-to-speech directly from the browser, stream audio into an avatar component, and ship it. For prototypes, that can work. For production, it often creates a security and operational problem that is easy to miss until you have real users, real billing, and real abuse.

This post is about deciding where TTS should live in an avatar app: frontend, backend, or a managed embed. By the end, you should be able to reason about API-key exposure, rate limiting, origin boundaries, and which architecture fits a realtime avatar experience without weakening your security model.

Why “just do it in the frontend” is usually the wrong default

The browser is an untrusted environment. That sounds obvious, but it is easy to forget once you have a polished UI and a working demo. If your frontend needs to call a TTS provider directly, you usually need credentials in the browser or a way to mint them from the browser. Either way, the browser becomes part of your trust boundary.

The core issue is not “frontends are insecure” in a vague sense. It is specific:

• Secrets leak. Any long-lived API key shipped to the browser can be extracted.

• Ephemeral tokens still need a minting path. If the browser can request unlimited short-lived tokens, your backend still needs abuse controls.

• Usage can be replayed. Network logs, local storage, browser extensions, and compromised devices all increase exposure.

• Rate limiting is harder. Once the browser is in the critical path, your enforcement becomes distributed and easier to bypass.

For avatar apps, the impact is larger than a normal SaaS API. TTS is often part of a realtime loop: user speaks, agent responds, audio is synthesized, the avatar lip-syncs, and the session continues. If an attacker can trigger synthesis cheaply or reuse your credentials, they can burn through quota fast. If the synthesis request also controls voice selection or prompt content, they may cause unintended behavior in the agent layer too.

Security model: what you are actually protecting

When developers ask whether TTS should be called from the frontend, they often focus on the API key. That is necessary, but not sufficient. You should think about three separate assets:

1. Credentials — API keys, session tokens, service accounts.

2. Policy — who is allowed to synthesize, what voice can be used, what content can be spoken, and how much can be spent.

3. Runtime state — active avatar sessions, conversation context, and the relationship between the audio stream and the visual stream.

A backend-mediated architecture keeps those assets server-side. The browser requests a session or playback event, but the server decides whether that request is valid. That gives you a clean place to enforce:

• authentication and authorization

• per-user and per-tenant quotas

• origin checks

• abuse detection

• content policy and prompt constraints

Direct-from-browser TTS can still be acceptable in narrow cases, but only if you are comfortable treating the TTS credentials as effectively public and have a compensating control model that does not depend on secret storage in the client. In practice, that is rare for production avatar apps.

Frontend TTS, backend TTS, and the trade-offs that matter

There are three common patterns.

1. Direct browser-to-TTS

The browser calls the synthesis service directly and either renders audio locally or passes audio to an avatar component. This is the simplest to wire up and often has the lowest initial latency because there is one less hop.

It also has the weakest security posture:

• credentials are exposed or indirectly exposed

• you need to trust client-side enforcement for quota and policy

• abuse prevention becomes difficult

• multi-tenant billing attribution is harder to make reliable

This pattern is usually only reasonable for internal tools, demos, or purely public/non-secret APIs where the cost of misuse is negligible.

2. Browser calls your backend, backend calls TTS

This is the usual production pattern. The browser sends a user action to your server, and the server performs synthesis using its own credentials.

That gives you a clear security boundary. A typical implementation looks like this:

import os

resp.raise_for_status()

import os

resp.raise_for_status()

import os

resp.raise_for_status()

The exact request shape will depend on the API endpoint you are using, but the important part is architectural: only your backend sees the credential. The browser gets back a session reference, an audio URL, a stream handle, or another limited artifact.

The downside is that your backend becomes responsible for realtime behavior. If you are synthesizing many short utterances in rapid succession, you need to manage concurrency, buffering, and retry behavior carefully. For avatar apps, this backend is not optional; it is part of the product.

3. Managed browser embed with server-side policy

There is a third option that is often better than rolling your own frontend integration: use a customer-managed iframe embed for the avatar surface. In this model, the avatar runs in a constrained embedded context, with allowlisted parent origins, per-embed voice and instruction settings, and rate limits enforced by the service rather than by the browser.

This is a better fit when you want to add an interactive avatar to a website without exposing a backend or API key in the browser. The iframe boundary is not a magic shield, but it gives you a concrete trust boundary and a simpler operational story than “let the frontend call everything directly.”

What changes for realtime avatars specifically

Realtime avatar apps are not just “audio plus video.” You usually have at least four moving parts:

• the user’s microphone or text input

• the language model or agent logic

• the TTS system

• the avatar renderer that needs synchronized speech timing

In a WebRTC-style or streamed avatar flow, the audio and the visual stream must stay aligned closely enough that lip sync looks natural. That means your request path, buffering, and session lifecycle matter. If you let the browser call TTS directly, you often end up with split-brain state: the avatar component knows about audio playback, while your backend still needs to know about billing, session ownership, or moderation.

That split becomes painful when you need to answer simple questions like:

• Which tenant spent this budget?

• Which session should be cut off after 10 minutes?

• Which origin is allowed to launch this avatar?

• Did this utterance come from a valid authenticated user or a copied browser script?

Those are backend questions. The more closely TTS is coupled to avatar state, the more important it is that the backend owns the synthesis request and session policy.

Where Protoface fits

This is exactly the kind of boundary Protoface is designed to make easier. If you are integrating a voice agent with a synchronized talking face, the cleanest path is usually to keep credentials and session management server-side, then connect the avatar layer through the appropriate surface for your stack.

For example, if you are already running a voice agent in LiveKit, the plugin and integration examples are a practical way to attach a realtime avatar without pushing secrets into the browser. If you are managing avatars or sessions directly, use the REST API or Python SDK from your backend so your application, not the frontend, owns authentication and policy. The docs at docs.protoface.com are the right place for the exact request and session shapes.

That approach lines up with the security model above: the browser can initiate or display the experience, but it should not hold durable credentials or make trust decisions about billing-sensitive synthesis.

Practical decision guide

If you are deciding between frontend and backend TTS, use this rule of thumb:

• Use frontend TTS only for prototypes, demos, or public APIs where misuse is acceptable and no secret can be harmed.

• Use backend TTS when you care about tenant isolation, quotas, policy enforcement, or reliable billing.

• Use an iframe embed or a server-driven avatar integration when you want a production-ready avatar surface without exposing credentials in the browser.

Also remember that “frontend” and “backend” are not just deployment choices; they are trust boundaries. If your architecture requires the browser to enforce what only the server can verify, you have already lost the argument.

Conclusion

Directly calling TTS from the frontend is tempting, but for avatar apps it usually weakens the parts of the system you care about most: credentials, quotas, session integrity, and abuse resistance. The safer default is to keep TTS and avatar-session authority on the backend, then let the browser render the result.

If you are building a realtime avatar experience, start with the architecture that matches your security model rather than the one with the fewest lines of code. Review the docs, pick the integration surface that fits your stack, and keep secrets out of the browser unless you have a very good reason not to.