Introduction

Realtime AI avatars usually fail for the same boring reason: not the model, not the lip-sync, but the network path. If you’re shipping a voice agent with a synchronized video face, the media connection needs to establish quickly, survive NATs and corporate firewalls, and keep latency low enough that the avatar feels attached to the conversation instead of lagging behind it.

This post is about configuring ICE servers correctly for that path. By the end, you should be able to choose the right STUN/TURN setup, understand why WebRTC connection timing matters for avatars, and verify your configuration in Python and TypeScript without cargo-culting a random config from another project.

What ICE is doing in a realtime avatar stack

ICE (Interactive Connectivity Establishment) is the mechanism WebRTC uses to find a workable network route between peers. In practice, it combines:

• STUN to discover your public-facing address and test direct connectivity.

• TURN to relay media when direct peer-to-peer paths fail.

For an avatar, “works eventually” is not enough. You care about:

• Connection setup time — the user should not wait several seconds before the face appears.

• Reliability under NAT — mobile networks, enterprise Wi-Fi, and symmetric NATs are common.

• Media stability — once connected, the stream should avoid sudden failures that break the illusion of a live presence.

There’s a useful mental model here: STUN is the cheap first try, TURN is the fallback that costs more but gets you through hostile networks. For production avatars, you generally want both available. If you only configure STUN, some percentage of users will never establish a usable media path. If you only configure TURN, everything works more often, but you pay for the relay and may add avoidable latency.

Choosing ICE servers: the practical trade-offs

The right ICE config depends on where your clients run and how sensitive your application is to latency and cost.

Use STUN when:

• You want to minimize relay usage.

• Your users are mostly on residential networks or mobile devices.

• You can tolerate some connection failures and retry behavior.

Use TURN when:

• You need predictable connectivity behind restrictive firewalls.

• You support enterprise customers or in-browser experiences on managed networks.

• You want a higher success rate for realtime avatar sessions, even if it means more relay traffic.

A few details matter a lot in practice:

• Transport: TURN over UDP is usually the first choice for media latency. TURN over TCP or TLS can help when UDP is blocked, but it is often a fallback path rather than the ideal path.

• Credential lifetime: long-lived static TURN credentials are convenient but less secure. Ephemeral credentials are better when you control the backend flow.

• Region: if your avatar service is regional, put ICE infrastructure close to your media servers and your users. TURN traffic is expensive in both bandwidth and latency when the relay is far away.

• Redundancy: provide multiple ICE servers. Browsers will try them in order, and you want at least one path that survives failure of a single provider or region.

For avatar applications, connection strategy often ends up looking like: STUN first, TURN second, with an aggressive timeout on connection establishment and a retry path if the initial route fails. The exact values depend on your frontend runtime and the media stack behind it, but the principle is consistent.

Python: configuring ICE servers for a session

If you’re creating sessions from Python, the main thing to get right is that the server-side session metadata contains the ICE configuration your client or embedded media stack will use. The exact request fields vary by SDK and docs version, so treat the snippet below as illustrative rather than copy-paste final.

from protoface import ProtofaceClient

print(session.id)

from protoface import ProtofaceClient

print(session.id)

from protoface import ProtofaceClient

print(session.id)

The important part is not the exact constructor shape; it’s the structure of the ICE config:

• urls points to one or more STUN/TURN endpoints.

• TURN entries usually need credentials.

• Multiple servers can be supplied so the browser can fail over.

If you are generating ephemeral TURN credentials on the backend, do that as close to session creation as possible. Do not bake credentials into frontend code. If your avatar sessions are browser-based, the client should receive only the minimum connection data needed to establish media, not privileged API keys.

TypeScript: passing ICE servers into a browser client

In TypeScript, the same logic applies whether you’re wiring a custom WebRTC client or integrating through a browser-facing SDK. The browser’s RTCPeerConnection takes an iceServers array directly.

const pc = new RTCPeerConnection({
});

const pc = new RTCPeerConnection({
});

const pc = new RTCPeerConnection({
});

If you are creating your own session bootstrap flow, have your backend return the ICE server list alongside the session descriptor, then feed it into the browser connection config. That keeps credentials off the client until they are needed, and lets you rotate TURN credentials without redeploying frontend code.

A practical debugging tip: inspect the selected candidate pair in chrome://webrtc-internals. If you see the connection repeatedly choosing relay when you expected direct candidates, that is often a sign of NAT restrictions, misconfigured STUN, or a TURN server that is being used as a fallback more than you intended.

Gotchas that matter for avatar latency

1. Don’t treat TURN as an afterthought. If your avatar must work for real users, you need a tested relay path. “It works on my home network” is the wrong success criterion.

2. Keep the ICE list small but redundant. Throwing a dozen servers into the config rarely helps. A few well-chosen endpoints, ideally with geographic and transport diversity, is better than a long list that slows initial candidate gathering.

3. Watch for credential expiration. Browsers may establish the connection after a delay, especially on poor networks. If your TURN credentials expire too quickly, you can get intermittent setup failures that are hard to reproduce.

4. Match timeout policy to the product. A conversational avatar usually benefits from a shorter “give up and retry” window than a background sync app. Users notice a dead face faster than a delayed analytics update.

5. Measure in the real network conditions you support. Test on office Wi-Fi, tethered mobile, VPNs, and a few geographically distant regions. ICE issues are often environment-specific.

How Protoface fits into this

For a developer-facing avatar platform, the main value is that you do not have to invent your own media plumbing just to attach a face to a voice agent. Protoface exposes the session and avatar controls you need, and the docs cover how connection details are surfaced for the relevant integration path.

If you are using the LiveKit Agents plugin, the cleanest mental model is: your voice agent handles speech and turn-taking, while the avatar layer needs a stable WebRTC connection with sensible ICE behavior so the video face stays synchronized. The plugin and quickstarts in the GitHub organization are useful references when you want to see the integration end to end. For API-level session creation and connection parameters, check the docs and wire your ICE configuration into whatever backend creates the session.

The practical benefit is that you can keep the sensitive parts of the flow on the server, generate connection settings per session, and avoid exposing API credentials in the browser while still giving users a fast connection path.

Conclusion

Fast, stable avatar connections are mostly about disciplined ICE configuration: provide STUN for cheap path discovery, TURN for reliability, keep credentials ephemeral when you can, and test under the networks your users actually have. If your avatar disappears or stalls, do not assume the model is the problem; check the media path first.

Once you have a working baseline, tune for your product’s priorities: lower latency, higher success rate, or lower relay cost. Then validate with real browsers and real networks. If you need implementation details for your specific integration path, start with docs.protoface.com and the relevant quickstart or plugin example for your stack.