Troubleshooting WebSocket and WebRTC Issues in a Browser-Based AI Receptionist
Debug WebSocket/WebRTC failures in browser AI receptionists: handshake, ICE/TURN, autoplay, and lifecycle issues.
Introduction
Browser-based AI receptionists fail in predictable ways: the audio path is fine, the model is responding, and then the user sees a frozen face, one-way audio, or a WebSocket that closes right after the session starts. In practice, most of these issues are not “AI problems” at all. They’re transport, signaling, autoplay, or network-policy problems.
This post walks through the failure modes I see most often when a voice agent is paired with a realtime avatar in the browser. By the end, you should be able to tell whether you’re dealing with a WebSocket handshake issue, a WebRTC media-path issue, a browser policy issue, or a bad integration between your agent and the avatar layer.
Start with the transport model: WebSocket is control, WebRTC is media
The first debugging mistake is treating WebSocket and WebRTC as interchangeable. They solve different problems:
WebSocket is usually the signaling/control channel: create session, negotiate parameters, exchange state, receive events, and push metadata.
WebRTC carries the realtime media: audio, video, and sometimes data channels. For an avatar, this is what drives the synchronized talking face in the browser.
If your WebSocket is healthy but the avatar is silent or frozen, the control plane is probably fine and the media plane is broken. If the WebSocket never stabilizes, you may never get far enough to establish the media session. Keep those layers separate in your head and in your logs.
Debug the WebSocket first: handshake, auth, and liveness
For browser apps, the most common WebSocket failures are usually boring:
401/403 during upgrade: wrong token, expired credential, or an origin policy issue.
101 switching protocols never happens: proxy or CDN not forwarding upgrade headers correctly.
Immediate close after connect: application-level auth failed, session parameters invalid, or the server rejected the requested avatar/session.
When debugging, inspect the handshake in the browser DevTools Network tab. You want to see the upgrade request, response headers, and the close code. A generic “WebSocket closed” message is not enough. Look for:
Sec-WebSocket-Acceptpresent in the response.No intermediary rewriting or buffering headers.
A sane close code, not just a network error.
If you are creating sessions from your backend, confirm the request itself is correct before involving the browser. A simple curl check is often faster than chasing frontend state:
The exact endpoint and payload depend on the resource you are creating; use the public docs for the current shape. The point is to verify that the API key, request body, and server-side permissions are valid before you debug the browser.
Then inspect WebRTC: ICE, DTLS, and media tracks
When the WebSocket is fine but the face never animates, the bug is usually in WebRTC negotiation or delivery. A browser may successfully connect to a realtime backend and still fail to receive media if:
ICE candidate gathering never completes because UDP is blocked or the network is too restrictive.
DTLS handshake fails due to incompatible network/security conditions.
The remote track is created but not attached correctly in the UI.
Autoplay policy prevents the browser from starting audio, which can indirectly stall your UX flow if your app waits on playback state.
For media debugging, use the browser’s WebRTC internals before you change code. In Chrome, chrome://webrtc-internals shows candidate pairs, ICE state, packets sent/received, jitter, and track events. The key states are:
new → checking → connected: normal path.
failed: connectivity problem, often TURN/firewall related.
disconnected: transient network loss; if persistent, treat it as a path issue.
If your deployment is behind corporate networks, hotel Wi-Fi, or restrictive mobile carriers, assume UDP may be unreliable. Your debugging question becomes: does the session work on a clean home network but fail in a more locked-down environment? If yes, start looking at TURN relay availability and firewall traversal, not your avatar rendering code.
Browser-specific problems that look like signaling bugs
Two browser behaviors frequently masquerade as transport failures.
Autoplay and user-gesture requirements. Chrome, Safari, and mobile browsers often require user interaction before playing audio. If your app initializes the agent and waits for audio playback state to advance, but you never clicked a button or otherwise triggered a gesture, the browser may block media start. That can make the whole session look broken even though the connection is fine.
Tab visibility and power-saving behavior. Background tabs can throttle timers and degrade realtime rendering. If the receptionist “freezes” only when the tab is backgrounded, verify that your app isn’t relying on a render loop or timers that are being suspended. The media stream may still be live while your local visualization stops updating.
In both cases, check the browser console for policy warnings. A clean WebSocket plus no visible avatar update often means the browser is refusing to play or render, not that the backend failed.
Make sure your signaling and media lifecycles match
Realtime apps often break because the app-level lifecycle and the transport lifecycle drift apart. A common example:
The frontend opens a WebSocket and requests a session.
The backend creates the session and returns connection data.
The frontend immediately tears down state because React re-renders or a route changes.
The WebRTC peer connection is orphaned, or the video element loses its stream reference.
The result is not a hard error; it’s a silent failure where the connection exists but nothing is attached to the page.
To prevent that, treat the session object, peer connection, and DOM attachment as distinct resources with explicit ownership. Keep a stable reference to the media stream and tear it down only when the session truly ends. If you are using React, avoid recreating the peer connection on every render. If you are using vanilla JS, make sure the same video element stays associated with the same stream for the life of the call.
Also watch for race conditions around “ready” events. Many stacks emit a session-created event before the remote tracks are actually flowing. If your UI assumes that “connected” means “video ready,” you can easily display an empty component for several seconds. A better pattern is to wait for the specific track-available or first-frame event before showing the avatar as active.
How I’d isolate the failure in practice
When a browser-based receptionist is broken, I’d reduce the problem in this order:
Confirm the backend session can be created outside the browser.
Confirm the WebSocket upgrade succeeds and stays open long enough to negotiate a session.
Confirm the WebRTC peer connection reaches
connected.Confirm the remote video track is attached to the right element.
Confirm autoplay, mute state, and browser policies aren’t blocking playback.
If you need to inspect the connection state in code, keep the logging simple and explicit:
That’s not fancy, but it tells you whether the session died before media arrived or after media arrived but failed to render.
Where Protoface fits in
When you’re using Protoface with a browser receptionist, the useful thing is that the avatar layer is exposed through standard developer surfaces: a REST API for session and avatar management, and browser-friendly embed patterns that avoid exposing API keys client-side. That matters because it lets you separate “can I create the session?” from “can the browser receive and display it?” instead of mixing authentication, media, and UI concerns into one fragile blob.
If your app needs a server-side integration, the REST API and Python SDK are the cleanest way to create and manage sessions from trusted code. If you’re embedding an avatar directly on a website, the customer-managed iframe model is especially helpful because the browser never sees your API key, and the embed can enforce origin allowlists and rate limits. For quickstarts and current request shapes, use the docs and the relevant repository such as the Python SDK or GitHub org examples.
For LiveKit-based voice agents, the plugin path is useful when the avatar should follow an existing realtime voice stack rather than replace it. In that setup, the debugging boundary is clearer: if the agent speaks but the face doesn’t move, focus on the plugin/session wiring and media delivery, not on the speech pipeline itself.
Conclusion
Most WebSocket and WebRTC issues in browser-based AI receptionists come down to a handful of root causes: bad signaling auth, proxy upgrade problems, ICE/TURN reachability, autoplay policy, or lifecycle bugs in the frontend. The fastest way to debug is to separate control-plane failures from media-plane failures and verify each layer independently.
Start with the handshake, inspect WebRTC state transitions, and confirm that the remote track is actually attached and allowed to play. Once you can reproduce the failure in one layer at a time, the bug usually becomes obvious. For implementation details, current endpoints, and integration examples, check docs.protoface.com.