Introduction

If you are building a Next.js app that streams realtime audio to an LLM and wants a talking face on top of it, the hard part is not rendering video. The hard part is keeping your model provider, your avatar provider, and your browser app separated by trust boundaries that make sense.

The common failure mode is simple: you put an ElevenLabs or OpenAI TTS key in the browser so the client can synthesize speech directly, then discover that anyone can extract it and spend your quota. The correct pattern is to keep provider credentials server-side, proxy only the minimum data needed for the session, and isolate avatar rendering behind a service that can consume audio/video in realtime.

By the end of this post, you should be able to:

• identify where TTS credentials should and should not exist in a Next.js realtime avatar stack,

• design a secure server-side proxy for ElevenLabs or OpenAI TTS,

• understand the latency and streaming implications of piping synthesized audio into a realtime avatar, and

• see where Protoface fits when you want to add a synchronized avatar without exposing any API keys to the browser.

The security boundary: browser, your server, and third-party APIs

For a realtime avatar app, there are usually three distinct paths:

1. Browser UI: microphone capture, playback, transcript display, and WebRTC signaling if you are doing live media.

2. Your backend: session orchestration, auth, prompt assembly, rate limiting, and any secret-bearing calls.

3. Provider APIs: LLM, STT, TTS, and avatar/video services.

Security problems happen when these boundaries blur. In practice, the browser should never receive provider API keys for OpenAI or ElevenLabs. It can receive short-lived session tokens minted by your backend, but those tokens should only authorize the specific user session and specific operations you intend.

For TTS, the safest default is:

• browser sends text or an assistant response event to your Next.js route handler,

• route handler calls the TTS provider with the secret key from environment variables,

• backend streams audio bytes back to the browser or forwards them to your avatar pipeline.

This keeps key material out of client code, browser storage, and network logs exposed to the user.

Why “just call TTS from the client” is the wrong abstraction

It is tempting to make the browser directly invoke ElevenLabs or OpenAI TTS because that looks lower-latency. The problem is that the browser is not a trust boundary you control. Even if you hide the key in an environment variable during build, any secret that ships to the client is recoverable. If you use a public proxy endpoint without auth, anyone can script it. If you use a signed token with broad scope and a long expiry, you have only moved the problem.

There is also a more subtle issue: TTS output is often not the final artifact. In an avatar app, speech audio usually feeds one of these paths:

• playback in the browser,

• WebRTC transport to a live avatar/session service, or

• a media pipeline that synchronizes audio with lip movement and facial timing.

That means you want a backend that can enforce policy before audio is emitted. For example:

• truncate or redact unsafe text before synthesis,

• limit per-user synthesis duration,

• pin model/voice parameters to a whitelist,

• attach request IDs for tracing and billing.

Once you let the browser hit the TTS vendor directly, those controls become much harder to enforce consistently.

A practical Next.js pattern: route handler as a TTS gateway

In a Next.js app router setup, the usual secure pattern is an API route that accepts authenticated requests from your frontend, then performs the vendor call server-side. If the provider supports streaming, your route can stream the response bytes onward instead of buffering the entire audio file.

import { NextRequest } from 'next/server';

}

import { NextRequest } from 'next/server';

}

import { NextRequest } from 'next/server';

}

The same shape works for OpenAI TTS: the client sends text to your route, the route calls OpenAI with process.env.OPENAI_API_KEY, and the response streams back. The details differ by SDK and endpoint, but the trust model does not.

A few implementation details matter:

• Use server-only env vars. Never prefix the provider key with NEXT_PUBLIC_.

• Prefer streaming when the provider supports it. Realtime avatars are latency-sensitive; buffering an entire utterance adds visible delay.

• Authenticate the route. Your proxy is only secure if the backend can distinguish a real user from a random script.

• Set hard limits. Cap text length, request rate, and maximum synthesis duration.

Streaming audio into a realtime avatar pipeline

If the avatar is expected to speak while the user is still interacting, the important property is not “TTS works,” but “audio arrives early enough to preserve conversational turn-taking.” In realtime systems, a few hundred milliseconds matter. The pipeline usually looks like this:

1. the agent decides it should speak,

2. text is generated incrementally or in a single response,

3. TTS begins producing audio as soon as enough text is available,

4. the avatar session consumes the audio and generates lip-synced video.

That means you should avoid unnecessary serialization points. Common mistakes include:

• waiting for the whole LLM response to finish before starting TTS,

• writing audio to disk before forwarding it,

• introducing a serverless function timeout shorter than the utterance length,

• proxying through the browser when the browser could just receive an already-authenticated session token.

If you are using WebRTC for media transport, remember that the browser is typically only one endpoint in the session. The server or agent runtime can hold the media connection and keep the user-facing app focused on UI and session orchestration. That separation makes it easier to keep secrets off the client and to swap TTS providers later.

Where Protoface fits: keep the avatar side server-managed

This is the part where using a dedicated avatar service helps. With Protoface, you can keep your TTS credentials on your side while handing the avatar/video synchronization problem to the avatar layer. If you are already running a LiveKit voice agent, the quickstart and the LiveKit plugin show the common pattern: the agent produces audio, and the plugin drops a synchronized talking face into the session without exposing anything to the browser.

A minimal Python-side integration looks roughly like this:

from protoface import ProtofaceClient

print(session.id)

from protoface import ProtofaceClient

print(session.id)

from protoface import ProtofaceClient

print(session.id)

For a live voice agent, your backend or agent runtime can keep the TTS step private, then forward the synthesized audio into the avatar session. The browser only needs the session-specific data required to render or connect; it does not need your OpenAI or ElevenLabs key. If you want the full integration path, the public docs are the right reference point for session fields, auth, and rate limits: docs.protoface.com.

One useful mental model is that the avatar service should be treated like media infrastructure, not like a frontend widget. That distinction matters because media infrastructure belongs behind server-authenticated APIs, while widgets tend to invite credential leakage.

Additional guardrails that actually matter in production

Once you have the basic proxy in place, most of the real work is operational:

• Per-user quota: limit synthesis minutes or character count per session.

• Idempotency: dedupe repeated requests if the browser retries after a timeout.

• Observability: log request IDs, upstream latency, and provider status codes, but never raw secrets.

• Fallback behavior: decide what happens if TTS is slow or unavailable. In an avatar app, silent failures are especially bad because the face continues to look “ready” while audio stalls.

• Prompt and content policy: if user input can trigger speech, validate it before synthesis to avoid generating audio you do not want to pay for or ship.

If you are switching between ElevenLabs and OpenAI TTS, keep the interface in your app narrow: something like synthesize({ text, voice, format }). That lets you change providers without reworking the avatar code path. The backend should own provider selection, not the browser.

Conclusion

The core rule is straightforward: never let browser code hold your TTS credentials, and never couple avatar rendering to secret-bearing client logic. Put ElevenLabs or OpenAI TTS behind a server-side route, stream audio where possible, and keep session-specific authorization narrowly scoped. If you are adding a talking face to a voice agent, treat the avatar layer as part of your realtime media stack, not as a frontend flourish.

If you want to implement this with a smaller surface area, start from the docs and a quickstart, then wire the model, TTS, and avatar pieces together behind your Next.js backend. The references at docs.protoface.com and the public examples in GitHub are the fastest way to sanity-check your architecture before you ship.