Inspecting WebRTC Stats for Realtime Avatar Performance Problems in SFU Deployments
Learn to inspect WebRTC getStats() in SFU avatar pipelines to debug loss, jitter, bitrate, decode, and frame pacing issues.
Introduction
When a realtime avatar looks “fine” in one environment and then suddenly lags, desyncs, or drops frames in production, the root cause is usually not the model. In SFU-based WebRTC deployments, the failure mode is often in the transport: packet loss, jitter, bandwidth estimation, decoder pressure, or a sender that is producing media faster than the downstream path can actually sustain.
This post is about reading getStats() like an engineer, not like a dashboard tourist. By the end, you should be able to inspect a session, identify whether the problem is capture, encode, uplink, SFU relay, or decode/render, and know what metrics matter for a realtime avatar pipeline.
Start with the transport path, not the symptom
A talking avatar is just a media pipeline with strict timing requirements. Audio drives the conversational cadence, while video carries the face that makes the agent feel present. In an SFU deployment, the sender publishes tracks to the SFU, the SFU forwards them to subscribers, and each browser or app endpoint decodes and renders what it receives.
That means there are multiple places where “the avatar looks broken” can originate:
Capture/production: the avatar renderer or compositor is not generating frames consistently.
Encoding: the encoder is overloaded, producing late frames, low bitrate, or unstable frame pacing.
Uplink: the sender cannot sustain the bitrate or suffers loss/jitter before the SFU.
SFU forwarding: the SFU may be fine, but a subscriber path is constrained by bandwidth adaptation or downstream loss.
Decode/render: the receiver is getting packets but cannot decode smoothly, often because of device load or resolution mismatch.
The important point: do not start by guessing “video quality problem.” Start by locating the stage where timing first goes bad.
What to inspect in WebRTC stats
The browser’s WebRTC stats API gives you per-connection and per-track telemetry. For avatar debugging, the useful metrics usually fall into a few buckets:
Frames and pacing: frames sent/received, frames dropped, frames decoded, total decode time, total encode time.
Network health: packets lost, jitter, round-trip time, available outgoing bitrate, retransmitted packets.
Bitrate and resolution: actual outbound/ inbound bitrate, frame width/height, frames per second.
Quality adaptation: NACKs, PLI/FIR, target bitrate changes, quality limitation reasons.
For an avatar, you want to answer a few concrete questions:
Is the sender actually producing frames at the intended cadence?
Is the sender encoding fast enough, or is encode time spiking?
Is loss or jitter causing the SFU path to fall behind?
Is the subscriber decoding what it receives, or are frames being dropped on the client?
If the answer to 1 and 2 is yes but 3 or 4 is no, the problem is transport or client-side playback, not avatar generation.
Reading stats in the browser
The practical way to inspect stats is to poll them periodically and correlate them with the user-visible symptom. For video, you usually want to inspect the outbound RTP stats on the publisher side and the inbound RTP stats on the subscriber side.
A few interpretation rules help a lot:
High framesEncoded, low effective motion often means the avatar is being throttled after encoding, or the receiver is dropping.
Rising packetsLost and jitter usually means network instability, not a rendering bug.
Quality limitation reason = bandwidth means the encoder or browser is adapting because the path cannot sustain the stream.
FramesDropped increasing on inbound means the client is receiving data but cannot decode or schedule it smoothly.
If you need one mental model: outbound stats tell you whether you are successfully sending a good stream; inbound stats tell you whether someone else can use it.
What SFU deployments add to the picture
SFUs change the debugging process because the server is not a media endpoint in the same sense as an MCU. It forwards selectively, often adapting layers, and it can hide problems by making one participant look healthy while another is struggling.
For realtime avatars, this matters because the avatar stream is often more visually sensitive than normal camera video. Faces expose stutter quickly: a missed frame on a static slide deck is annoying; a missed frame on lip motion looks broken.
In an SFU setup, compare stats on both sides of the pipe:
Publisher side: verify the avatar publisher is maintaining stable frame pacing and bitrate.
Subscriber side: verify the client is receiving enough packets and decoding without drops.
Cross-check timing: if publisher stats look clean but subscriber stats show loss or drops, the problem is downstream of the publisher.
One useful metric is round-trip time paired with jitter. Rising RTT alone does not always cause visible issues, but RTT plus loss often triggers bitrate reduction, which then makes avatar motion look soft or delayed. Another useful signal is frequent keyframe requests. If the receiver keeps asking for keyframes, the stream may be unstable enough that the decoder cannot recover cleanly.
How to debug systematically
A reliable workflow is:
Reproduce with a short session. Long sessions make it harder to see the onset of the problem.
Sample stats every second. Averages hide spikes; spikes are often the cause.
Correlate with application events. Note when the avatar starts speaking, when the network changes, or when the user switches tabs.
Check both audio and video. Audio can remain intelligible while video degrades, which masks the transport issue.
Separate sender from receiver. If you can test the same stream on two clients, compare the inbound stats.
Common failure patterns and what they usually mean
Here are the patterns I see most often in realtime avatar systems:
Video stutters but audio is fine: video bitrate is being reduced, frames are dropped on decode, or the avatar renderer is not pacing frames consistently.
Everything looks delayed by a second or two: congestion control is backing off, packets are retried, or the client is buffering due to loss.
Motion is smooth at low resolution but breaks at higher quality: the path cannot sustain the higher tier, or the receiver cannot keep up with decode cost.
Frequent keyframe requests: packet loss is making the decoder lose reference frames, or the subscriber is joining late and never stabilizing.
For avatar workloads, also remember that the visual content is unusually face-centric. Human observers notice mouth timing mismatches quickly. A stream that is technically “acceptable” can still be perceived as broken if lip motion lags even a few hundred milliseconds behind speech.
Where Protoface fits
If you are integrating avatars into a LiveKit voice agent, the LiveKit-oriented integration path is the cleanest place to surface stats alongside the avatar session. The plugin drops a Protoface avatar into the agent so the same debugging workflow applies: inspect the underlying WebRTC connection, then decide whether the issue is transport, decode, or avatar generation.
That matters because the avatar is not a separate magic stream; it is still subject to the same WebRTC constraints as any other realtime media. In practice, I recommend logging the session identifiers from your app and correlating them with WebRTC stats timestamps so you can line up user reports with transport events. If you need to create or inspect sessions programmatically, the REST API and Python SDK are the right tools, and the exact request fields live in the docs.
For quick local experimentation, the developer docs and the quickstarts are enough to wire a test session end-to-end. If you are already on LiveKit, the plugin repository is the most relevant starting point: https://github.com/protoface-ai/protoface-plugin-pipecat. If you need broader implementation details, the documentation is the authoritative reference.
Conclusion
In SFU deployments, realtime avatar bugs are usually observability bugs first. WebRTC stats let you determine whether the issue is in generation, encoding, network transport, or decoding, instead of treating all latency as the same problem.
Use the browser stats API to track frame pacing, loss, jitter, bitrate, and decode health. Compare publisher and subscriber views. Watch for the point where the metrics first diverge from the expected behavior. Once you can name that failure stage, the fix is usually straightforward: reduce bitrate, address packet loss, change resolution, or move the rendering workload.
For implementation details, integration examples, and session APIs, start with the docs at docs.protoface.com.