How to Use the Protoface REST API for ICE Candidate Exchange in a Realtime Avatar App
Use Protoface REST API to create realtime avatar sessions and exchange ICE candidates for WebRTC signaling.
Introduction
In a realtime avatar app, the hard part is rarely rendering the face. The hard part is keeping the media pipeline stable while your agent is negotiating WebRTC, exchanging ICE candidates, and recovering cleanly when network conditions change.
This post focuses on that negotiation layer: how to use the Protoface REST API to create and manage a session, exchange ICE candidates, and connect a browser or backend client to a realtime avatar. By the end, you should understand the signaling flow well enough to wire it into your own app, debug failed connections, and know where the REST API fits relative to the SDK and other integration surfaces.
What ICE exchange is doing in this system
WebRTC media does not start flowing until both peers know how to reach each other. ICE, or Interactive Connectivity Establishment, is the process of discovering candidate network paths and selecting the one that actually works between the client and the avatar session.
In practice, your app will:
Create a realtime session on the server.
Generate or receive an SDP offer/answer depending on your client flow.
Collect local ICE candidates as the browser or agent discovers them.
Send those candidates to the remote side via signaling.
Receive remote candidates and add them to the peer connection.
Wait for ICE to connect, then start receiving synchronized avatar video and audio.
The REST API is the place to manage that session state if you are not using a higher-level SDK. The important thing to keep in mind is that REST is not the media plane. It is the control plane that helps you create the session, store identifiers, and exchange signaling messages around the WebRTC connection.
Session lifecycle: create first, then signal
The first thing you need is a server-side authenticated request using your API key. Keep the key out of the browser; the browser should only ever talk to your own backend or to a customer-managed embed when that is the intended product shape.
At a high level, the flow looks like this:
The exact request and response fields are in the docs, but the shape is always the same: create a session, get back identifiers and negotiation data, and use those values to initialize your WebRTC client.
A useful implementation detail: treat the session as ephemeral. If the client refreshes, the network drops, or you need to restart the agent, create a new session rather than trying to “resurrect” a stale peer connection. That makes retry logic much simpler and avoids a lot of half-open signaling state.
Exchanging ICE candidates with the REST API
Once the peer connection is created, the browser or backend will begin emitting local ICE candidates. Those are the server-reachable paths your client has discovered, such as host candidates on a local network, server-reflexive candidates through STUN, and relay candidates when TURN is needed.
You then forward each candidate to the remote side through your backend signaling flow. A typical REST-driven exchange looks like this:
And when the remote side returns its own candidates, you add them to the local peer connection with the equivalent WebRTC API on the client side.
That example is intentionally generic. The exact JSON shape depends on the API contract in the docs, but the important pattern is stable: your frontend collects candidates, your backend authenticates and forwards them, and the session endpoint stores or relays them to the other peer.
Practical WebRTC gotchas that matter here
Most ICE failures in realtime avatar apps are not “avatar” problems. They are networking or signaling problems. A few things to check first:
Candidate timing: don’t assume the SDP exchange finishes before ICE starts. Your signaling handler should tolerate candidates arriving before or after the offer/answer is set.
Trickle ICE: send candidates as they are discovered instead of waiting for gathering to complete, unless the docs or your transport require batching.
TURN fallback: if users sit behind restrictive NATs or enterprise firewalls, relay candidates may be the only viable path.
State transitions: watch
iceConnectionStateandconnectionState. “Checking” is normal; “failed” means you should retry with a fresh session.Media tracks: the avatar video track can connect independently of your app’s mic capture or other data channels. Debug each track separately.
For debugging, log the full sequence: session creation response, SDP offer/answer IDs, each candidate exchanged, and the final ICE state. When a connection fails, that transcript is usually enough to identify whether the issue is authentication, signaling, or network reachability.
Using the Python SDK for server-side orchestration
If you are creating sessions from a backend service, the Python SDK is often the cleanest way to keep the auth and request shapes out of application code. It is also a good fit when your app needs to create sessions on demand, attach custom instructions, or reconcile usage metadata with your own user model.
A minimal pattern looks like this:
From there, your backend can hand the session identifier and any signaling payload to the browser, which then performs the WebRTC negotiation. This division keeps credentials server-side and makes it much easier to rotate keys or apply rate limiting.
If you prefer to work from a typed SDK rather than raw HTTP, the Python package and examples are documented in the public repo and the main docs. Start with the SDK when you are building product logic; drop to REST when you need to inspect the exact wire format or integrate with a non-Python service.
Where Protoface fits in the stack
The REST API is the right tool when you need direct control over session creation and signaling. That is especially true if your application already has a custom auth layer, a backend relay, or a multi-step onboarding flow.
If you want the shortest path to a working avatar in a voice agent, the LiveKit plugin is usually the better abstraction. It handles the “give my agent a synchronized face” part so you can focus on the agent logic instead of the media plumbing. The same general media concepts still apply underneath, but you spend less time wiring candidate exchange by hand.
For teams building a larger integration, the docs at docs.protoface.com are the best reference for exact request/response fields, session lifecycle details, and any limits that affect your transport design. If you want to see the Python pieces in context, the SDK repository is also useful: github.com/protoface-ai/protoface-sdk-python.
Recommended implementation pattern
If you are building this into a realtime avatar app from scratch, a sensible architecture is:
Your frontend creates an RTCPeerConnection and gathers local ICE candidates.
Your frontend sends signaling messages to your backend.
Your backend authenticates with the Protoface API key and creates a session.
Your backend forwards candidates and any session metadata to Protoface.
Your frontend applies the remote description, adds remote candidates, and waits for ICE to connect.
That split gives you one authoritative place for credentials, observability, and retry policy. It also keeps the browser logic focused on what it is good at: media capture, playback, and peer connection state.
Conclusion
ICE candidate exchange is the part of realtime avatar integration that most often determines whether the experience feels solid or flaky. The core idea is simple: create a session server-side, exchange SDP and ICE candidates through your signaling path, and let WebRTC establish the best reachable route for the media.
If you keep the session ephemeral, log candidate exchange carefully, and retry on failure with a fresh session, the rest of the avatar stack becomes much easier to reason about. For concrete request shapes, examples, and integration options, start with the docs and then move into the SDK or plugin that best matches your app.