Debugging WebSocket and WebRTC Drops in a Flutter Hospitality Agent
Debug Flutter WebSocket and WebRTC drops in realtime avatar apps: state logs, ICE stats, mobile lifecycle fixes.
Introduction
When a Flutter app hosts a realtime avatar over WebRTC, “it worked on my desk” often stops being meaningful. The failure modes are messy: a signaling socket drops while media keeps flowing, ICE connectivity succeeds and then the video freezes, the app is backgrounded, a mobile network changes, or the peer connection stays up but the avatar is effectively silent because the upstream audio track stalled.
This post is about debugging those failures systematically. By the end, you should be able to distinguish WebSocket issues from WebRTC issues, instrument the right layer, and apply the usual fixes for Flutter-based voice/agent clients without guessing.
Start by separating signaling from media
The first mistake is treating “the connection” as a single thing. In a realtime avatar or voice-agent app there are usually at least two independent channels:
Signaling: a WebSocket or HTTPS control path used to negotiate session state, exchange SDP, authorize the client, and coordinate lifecycle events.
Media: the WebRTC peer connection carrying audio, video, and data channels once ICE/DTLS/SRTP are established.
Those layers can fail independently. A WebSocket disconnect may prevent renegotiation or session refresh while an existing peer connection keeps running. Conversely, the WebSocket can stay healthy while the media path is dead because ICE connectivity failed, the TURN server is unreachable, or the remote peer stopped sending RTP.
In practice, you want explicit logs for both layers. Don’t just log “disconnected.” Log:
WebSocket close code and reason.
Peer connection state transitions:
iceConnectionState,connectionState,signalingState.Whether the local audio track is still producing frames.
Whether remote inbound RTP stats continue to increase.
Use browser- and device-level symptoms to narrow the failure
The most useful debugging trick is to classify the symptom before you touch code.
1. The WebSocket closes first
If signaling drops before media, the likely causes are boring but common: proxy timeouts, idle connection reaping, auth expiry, or a reconnect path that isn’t resubscribing to session state. In Flutter, check whether your websocket client sends keepalives and whether your backend or edge proxy closes idle connections after 30–60 seconds.
For long-lived sessions, you usually need one of these patterns:
Ping/pong heartbeats on the signaling socket.
Session resumption logic that reattaches to the same agent/avatar session after reconnect.
Short-lived tokens that can be refreshed before expiry, not after the socket is already gone.
If you only discover the closure in the UI, you’re already too late. Log the close event with a timestamp and compare it to your server-side session timeline.
2. The peer connection fails while signaling looks fine
This is the canonical WebRTC problem: signaling succeeded, but ICE never established a stable path, or it established one and then lost it. The state machine matters:
new/checkingstuck for a long time often means no viable candidate pair, blocked UDP, or broken TURN.connectedchanging todisconnectedmay be a transient network event.failedusually means the transport is done; your app should renegotiate or reconnect.
On mobile, frequent network transitions are a major source of trouble. Flutter apps can move between Wi-Fi and LTE, background and foreground, and device audio routing changes can interrupt capture even when the socket remains alive.
Inspect WebRTC stats instead of relying on the UI
When debugging media drops, stats are more reliable than eyeballing a frozen avatar. Collect a snapshot every few seconds and focus on a small set of fields:
candidate-pair: current transport selected, round-trip time, bytes sent/received.inbound-rtp/outbound-rtp: whether media bytes are moving.track: audio level, frames decoded, jitter buffer behavior.
If bytes stop increasing but the connection state remains “connected,” the issue is often above ICE: the sender stopped producing media, the track was muted, audio permissions changed, or an audio processing pipeline drained.
In Flutter, wire this up to periodic diagnostics. Keep the output terse but structured so you can correlate it with mobile OS events:
If your WebRTC package exposes stats collection, log the selected candidate pair and outbound/inbound byte counters. The exact API varies by package version, so treat the method names in the docs as authoritative.
Common Flutter-specific gotchas
Flutter itself is not the problem, but mobile lifecycle and plugin behavior amplify weak assumptions.
Audio capture stalls after backgrounding
On iOS and Android, background transitions can interrupt microphone capture or suspend the app enough that the sender appears alive but no fresh audio frames are sent. If the remote avatar stops lip-syncing first, verify that your local microphone track still exists and isn’t muted by OS policy.
For voice agents, this is especially easy to miss because the app may still play remote audio while the upstream speech path has died.
ICE restart is not automatic unless you make it so
A common misconception is that WebRTC will recover from every network change on its own. It often won’t. If the network changes materially, you may need to trigger an ICE restart or rebuild the peer connection. Design for this explicitly:
Keep a reconnect path that can re-establish media without a full app restart.
Preserve session identity separately from the peer connection object.
Make sure the signaling layer can request new SDP/ICE exchange cleanly.
Stale event listeners create phantom bugs
In Flutter, it’s easy to attach multiple listeners during widget rebuilds or fail to dispose them on navigation. That produces noisy logs that look like duplicate disconnects or random renegotiations. If your debugging session becomes inconsistent, verify ownership:
Who creates the websocket?
Who owns the peer connection?
Who disposes the stream subscriptions?
Can the same session be re-entered after route changes?
If the answer is “multiple widgets do,” refactor before chasing the network.
Reproduce with controlled failure modes
The fastest way to debug a flaky realtime session is to simulate the three likely failure classes independently:
Network loss: toggle airplane mode, switch Wi-Fi, or use a network conditioner.
Signaling interruption: kill the websocket path but keep the app process alive.
Media interruption: mute the mic, revoke permissions, or suspend audio capture.
Then observe which signals change first. If the app says “disconnected” but the peer connection state never moved, the bug is in your control plane. If the peer connection dies but the websocket stays up, look at TURN, ICE, or mobile lifecycle. If both remain nominal but the avatar freezes, inspect the sender pipeline and media stats.
A practical checklist for postmortems
For each dropped session, capture a single record with these fields:
Session ID and user/device ID.
WebSocket close code/reason and timestamp.
Peer connection state timeline.
Last successful RTP byte counters.
App lifecycle event at the moment of failure.
Network type before and after failure.
That’s enough to separate “backend auth expired,” “mobile network changed,” and “client stopped producing audio” in a few minutes instead of a few hours.
Where Protoface fits
Protoface is useful here because it gives you a clean boundary between your agent logic and the avatar/session machinery. If you’re integrating a voice agent, the LiveKit plugin keeps the avatar layer inside the agent runtime instead of forcing you to hand-roll video orchestration. If you’re creating or inspecting sessions directly, the REST API and Python SDK let you confirm that the session exists, is authenticated, and is in the state you expect before you chase transport issues. The docs at docs.protoface.com are the place to check exact session and avatar fields.
A minimal REST check looks like this:
And a Python SDK call might look like:
In a LiveKit agent, the plugin helps you keep the avatar synchronized with the agent’s audio output, which is exactly where many “it’s connected but not speaking” bugs show up. If you prefer to stay close to the agent runtime, start from the plugin examples in the repository rather than composing everything yourself: github.com/protoface-ai.
Conclusion
Debugging dropped realtime avatars is mostly about respecting the stack boundaries. WebSockets carry control. WebRTC carries media. Flutter adds lifecycle complexity. If you instrument each layer separately, log the state transitions, and collect basic WebRTC stats, the failure usually becomes obvious.
For implementation details, session fields, and current integration patterns, check docs.protoface.com. If you’re wiring an avatar into a voice agent, start with the relevant plugin or SDK example, reproduce the failure intentionally, and keep the postmortem data small but complete. That’s the difference between guessing and actually fixing the drop.