Reducing Realtime AI Avatar Latency: Optimizing ICE, STUN, and TURN for Faster Call Setup
Reduce realtime AI avatar call setup latency by tuning WebRTC ICE, STUN, and TURN with trickle ICE and better candidate handling.
Introduction
If your realtime AI avatar feels “slow,” the bottleneck is often not model inference. It’s usually the network path you create before media can flow: ICE candidate gathering, STUN connectivity checks, TURN fallback, and the time it takes a browser, mobile client, or agent backend to settle on a usable transport. For voice agents with a talking face, that extra second is very noticeable because users are waiting on both audio and video to become interactive.
This post focuses on the parts you can actually tune: how WebRTC connection setup works, where latency comes from, and how to reduce call setup time without sacrificing reliability. By the end, you should be able to reason about your connectivity path, identify when you are paying for TURN unnecessarily, and make practical changes that improve first-frame and first-audio times.
What ICE is doing during call setup
In a realtime avatar session, media usually rides over WebRTC. Before audio or video can move, peers need to find a route to each other. That process is ICE: Interactive Connectivity Establishment.
At a high level, ICE does four things:
Gathers local network candidates, such as host candidates from the device itself.
Uses STUN to discover server-reflexive candidates, which tell each side what public-facing address and port it appears to have.
Tries connectivity checks between candidate pairs to find a path that actually works.
Falls back to TURN relays if direct peer-to-peer or NAT traversal fails.
The important thing for latency is that ICE is not just one step. It is a search. The more candidate pairs you create, the more checks you can potentially run, and the longer it can take before the agent starts sending media. That delay is small on a good network and irritating on a bad one, especially when the avatar is already tied to an LLM and speech pipeline.
Where the latency really comes from
There are three common reasons setup feels slow:
Waiting for candidate gathering to finish. Some clients wait too long before starting signaling, or they only start the offer/answer exchange after all candidates are collected. Trickle ICE avoids that.
Over-relying on TURN. TURN is reliable, but relayed media adds an extra hop. That increases setup time and steady-state latency.
Poor network defaults. Conservative NATs, VPNs, enterprise firewalls, and mobile carrier networks can make the first few candidate pairs fail, so your client spends time probing bad paths before landing on the one that works.
For realtime avatars, the user experience often depends on time to first audio more than on perfect bitrate or ideal routing. If your call connects in 700 ms instead of 2.5 s, the product feels much more responsive even if the total session quality is similar.
How to optimize ICE, STUN, and TURN in practice
1. Use trickle ICE and start signaling early
Do not wait for all candidates before sending your offer or answer. Trickle ICE lets the peer exchange candidates incrementally, so the remote side can start checking earlier. That usually shaves meaningful time off setup, especially on clients with multiple network interfaces.
In practice, this means your signaling path should accept candidates as they arrive and pass them through immediately. If you are waiting to bundle everything into one blob, you are slowing down the connection for no real gain.
2. Keep STUN simple and close to your users
STUN is cheap, but it still costs round trips. Use a small number of well-placed STUN servers and avoid overcomplicating the configuration unless you have hard data that says otherwise. The goal is not to collect every possible candidate; it is to discover the ones that are most likely to work quickly.
For globally distributed users, a nearby STUN server reduces the time it takes to discover reflexive candidates. The practical effect is faster ICE negotiation, particularly for clients behind NATs that still allow direct UDP traversal once mapped correctly.
3. Treat TURN as a fallback, not the default
TURN is essential for reliability, but it is the slow path. If you route everything through TURN, you are paying relay latency for sessions that could have used a direct path. A better pattern is to prefer direct candidates and only fall back to TURN when ICE proves that direct connectivity is not viable.
There are two knobs worth understanding:
Candidate priority and ordering. Make sure your client prefers host and server-reflexive candidates before relay candidates when policy allows it.
TURN transport choice. If UDP is available, it usually performs better than TCP. TCP-based TURN may save you from firewall issues, but it adds latency and can amplify head-of-line blocking.
You should also keep TURN credentials short-lived and scoped appropriately. That is not just a security concern; it also makes operational behavior cleaner because clients only use relay when they truly need it.
4. Prune what you do not need
Every extra candidate pair expands the search space. If you know your deployment only needs a narrow set of transports, reduce the explosion. For example, some environments do not need IPv6 candidates, or they may benefit from disabling redundant interfaces on the client side. The more you can avoid probing dead ends, the faster the first successful pair will emerge.
Be careful here: over-pruning can reduce connectivity rates. The right approach is to measure. Start with the standard configuration, log candidate types and selected pairs, and only trim what your traffic proves is noise.
5. Measure time-to-connected, not just “call failed”
A lot of teams only notice connectivity when it breaks. That misses the more common problem: connections that technically succeed but feel slow. Track the following timestamps in your client or signaling layer:
offer created
first candidate gathered
remote description set
ICE connected
first audio packet sent
first video frame rendered
That gives you a breakdown of where setup time is actually going. If candidate gathering is slow, focus on STUN and client networking. If candidates arrive quickly but ICE connected takes too long, your issue may be candidate ordering, TURN fallback, or a bad network path.
A minimal WebRTC signaling pattern
Here is a stripped-down example of how trickle ICE typically looks from the application’s point of view. Exact wiring depends on your signaling server and media stack, but the pattern is the same: send the offer early, then forward candidates as they arrive.
If your implementation waits until all candidates are discovered before sending the offer, that is usually the first thing to change.
How this maps to Protoface in a real avatar stack
Protoface sits on top of the media path for developer-facing realtime avatars, so these same connection principles apply whether you are using a LiveKit voice agent plugin, the REST API, a Python SDK workflow, or an iframe embed. The difference is that you are not wiring the avatar logic itself; you are mostly making sure the session can establish a low-latency media path quickly.
If you are integrating a voice agent, the relevant lever is usually the WebRTC path between your client and the avatar session. Keep candidate gathering incremental, prefer direct connectivity when available, and treat relay as a fallback for difficult networks. The official docs at docs.protoface.com are the right place to confirm the exact session and signaling fields for your chosen integration surface.
For example, if you are using the LiveKit plugin, the fastest wins usually come from the underlying LiveKit/WebRTC configuration rather than from avatar-specific code. If you are using the Python SDK or REST API to create sessions programmatically, make sure your client-side WebRTC setup is not introducing extra delay before the avatar can join and start rendering.
Common gotchas
TURN everywhere in staging. Staging often runs behind corporate VPNs or restrictive NATs, which makes TURN look mandatory. Do not let that environment become your production default without measuring real user networks.
Late session creation. If you wait to create the avatar session until after the browser has already done several expensive setup steps, you add avoidable latency. Start the session flow as early as possible.
Ignoring IPv6. In some networks IPv6 works better than IPv4, and in others it is the opposite. Let your ICE stack measure rather than hard-coding assumptions unless you have concrete deployment data.
Not logging selected candidate types. Without that, you cannot tell whether users are mostly direct, srflx, or relay. This is one of the most useful metrics you can add.
Example: creating a session from the API side
If you are using the control plane to create or manage avatar sessions, keep the client path lightweight and let the media stack do its job. The control request itself is not the source of media latency, but it can gate when signaling starts.
Exact request and response fields depend on the API version and session shape in the docs, but the operational point is simple: create the session early, then let the media connection proceed without extra round trips in your app layer.
Conclusion
If a realtime avatar feels sluggish, inspect the path to media first. ICE, STUN, and TURN are not just background WebRTC plumbing; they are often the difference between a session that feels instant and one that feels hesitant. The practical recipe is straightforward: trickle candidates, keep STUN lean, use TURN as fallback, measure actual connection milestones, and avoid adding avoidable round trips before media can flow.
Once you have that instrumentation in place, you can make informed trade-offs instead of guessing. For implementation details and integration-specific guidance, check docs.protoface.com and the relevant examples in the developer repos linked there. If you are building on LiveKit or a Python workflow, those surfaces are usually where the biggest latency wins show up first.