Introduction

In a realtime AI avatar app, your TTS provider key is one of the few secrets that can immediately turn into cost, abuse, or data leakage if it lands in the wrong place. The common mistake is treating a browser embed, a client-side SDK, or a frontend demo like a safe place to call TTS directly. It isn’t. If the key can be extracted from shipped JavaScript, a network trace, or a misconfigured proxy, it will be used.

This post covers how to keep TTS credentials off the client, how to isolate them in a realtime avatar architecture, and how to choose the right boundary between browser, backend, and streaming media services. By the end, you should be able to design an avatar app where the browser never sees privileged API keys, the TTS layer is invoked server-side only, and rate-limited session tokens do the minimal work they need to do.

Start with the right boundary: the browser should never own the TTS key

The first decision is architectural, not cryptographic. If your app generates audio for an avatar in response to user text or agent output, the secret that authenticates TTS requests belongs on a server you control. The browser can initiate a session, send user input, and play streamed media. It should not be able to call the TTS vendor directly using a long-lived key.

Why this matters in practice:

• Anything in frontend code is effectively public.

• Even if you obfuscate a key, users can inspect network calls or bundle output.

• Realtime apps often have multiple moving parts: token minting, avatar session creation, media transport, and TTS generation. Mixing those responsibilities usually leaks secrets somewhere.

The safe pattern is:

1. The browser authenticates to your backend with your normal app auth.

2. Your backend decides whether the user can create a session or speak to the avatar.

3. Your backend calls the TTS service using the secret key.

4. The backend streams or returns audio to the realtime pipeline.

If you’re using a WebRTC-based avatar or voice-agent stack, that usually means the browser is only responsible for signaling and media playback, not vendor authentication. The actual voice synthesis should happen where you can log, throttle, and rotate credentials.

Use ephemeral tokens for the client, not API keys

What the client needs is not an API key; it needs a narrowly scoped, short-lived capability. That might be a session token, an embed token, or a signed URL that authorizes one specific action. The important property is that it expires quickly and cannot be reused as a general-purpose secret.

A useful way to think about it:

• API key: identifies and authorizes your server-side integration.

• Session token: authorizes one browser tab, user, or iframe session for a limited time.

• Media stream: carries audio/video after the session is established; it does not need your vendor secrets.

For TTS specifically, the browser can send a user utterance or an agent prompt to your backend, but your backend should do the vendor call and then feed the resulting audio into your realtime transport. If you proxy TTS through your server, keep the proxy strict: do not accept arbitrary vendor URLs or headers from the client, and do not let the client choose which secret is used.

Practical server-side TTS flow

A minimal pattern is a backend endpoint that accepts text, validates it, calls TTS with a secret stored in environment variables or a secret manager, and returns audio or an internal reference to that audio. The exact transport depends on your avatar stack, but the security boundary stays the same.

import os

return Response(r.content, mimetype="audio/mpeg")

import os

return Response(r.content, mimetype="audio/mpeg")

import os

return Response(r.content, mimetype="audio/mpeg")

This example is intentionally generic. In a real app, you may stream PCM, store the output temporarily, or inject it into your realtime avatar pipeline. The security principle is the same: the secret never leaves the server.

Harden the secret lifecycle: storage, rotation, and blast radius

Once the key is server-side, the next problem is operational hygiene. A secret that sits in a repo, a Dockerfile, or an unguarded config file is still a liability. Treat TTS credentials like any production secret:

• Store them outside the source tree: environment variables, secret manager, or runtime injection.

• Use distinct keys per environment: local, staging, production.

• Rotate regularly: especially after vendor changes, employee departures, or suspicious usage spikes.

• Scope by service: do not reuse one provider key for everything if the vendor supports narrower scopes.

• Log safely: log request metadata and timing, never raw Authorization headers or full request bodies if they may contain sensitive text.

For realtime systems, there is also a blast-radius issue. If a key is compromised, an attacker can often generate a large amount of speech quickly, which can turn into cost or abuse before you notice. Put rate limits on your own backend endpoint, even if the TTS provider also rate-limits. Defend at both layers.

One more gotcha: if you cache synthesized audio, make sure the cache key is based on normalized content and authorized usage. A public cache keyed only by text can accidentally let one user retrieve another user’s generated content.

Don’t leak secrets through logs, embeds, or “temporary” client code

The easiest place to lose a TTS key is not a production incident; it’s a quick prototype. Common failure modes include:

• Putting the key in frontend environment variables that are bundled into JavaScript.

• Logging request headers from a proxy or API gateway.

• Passing the key as a query string parameter to a media or synthesis endpoint.

• Using a browser-based demo that calls TTS directly “just for now.”

If you need a frontend to trigger synthesis, have it call your backend with an authenticated request, then return only the result the client needs. If you need multiple clients to request speech, issue per-session authorization tokens from your backend rather than sharing the TTS key broadly.

For realtime avatars, the separation should be even stricter when you have an iframe or embedded experience. The iframe should be able to join a session and exchange media, but it should not contain any vendor credentials. If your architecture requires cross-origin communication, use a narrow allowlist and short-lived session authorization, not a hidden API key.

How this fits in a realtime avatar stack

In a voice-agent or avatar application, the media path and the secret path are different systems. The media path handles audio/video transport, lip sync, and session state. The secret path handles vendor authentication, billing, and access control. Keeping those paths separate is what makes the system maintainable.

If you are wiring a voice agent into a talking face using the LiveKit plugin, for example, the plugin belongs on the server side with the rest of your agent stack. Your browser does not need the TTS key; it only needs to connect to the session and play media. The same applies if you use the Python SDK to create avatars or sessions: let backend code hold the API key and mint the exact runtime objects your app needs.

For implementation details, the docs are the right place to confirm the exact request and session shapes: docs.protoface.com. If you want a concrete integration example for a live agent, the plugin repository is also useful: GitHub examples for the Pipecat integration.

A sensible reference architecture

If you’re designing this from scratch, a good default is:

• Backend: stores TTS and avatar API keys, validates user identity, creates sessions, calls TTS, emits audio into the realtime pipeline.

• Client: authenticates to your backend, receives a short-lived session token or embed URL, renders the avatar, and plays audio/video.

• Realtime layer: transports media and session state, but never sees your TTS secret.

That model scales from a simple demo to a production support agent. It also makes audits easier: if you need to answer “who can generate speech, with which credentials, and from where,” the answer lives in your backend logs and secret manager, not in client code.

For a quick implementation path, the Python SDK can keep secret-bearing operations on the server while your frontend remains credential-free: Python SDK repository.

Conclusion

Securing TTS keys in a realtime avatar app is mostly about discipline: keep the key server-side, issue short-lived client tokens, validate and rate-limit requests, and never let the browser talk to TTS vendors directly. In realtime systems, “temporary” shortcuts tend to become permanent attack surfaces.

If you’re building an avatar app and want to verify the exact integration points, start with the docs and the quickstarts, then keep the secret-bearing code on the backend where it belongs: docs.protoface.com. From there, it’s straightforward to wire a voice agent, a web embed, or a custom backend without exposing your TTS credentials to the client.