Introduction

When a realtime avatar “echos,” “feeds back,” or starts sounding metallic, the problem is usually not the avatar model itself. It’s almost always a system-level audio bug: microphone capture leaking into playback, duplicate audio paths, unstable resampling, browser autoplay restrictions, or an agent loop that is hearing its own output and responding to it.

If you’re building a voice agent with a synchronized talking face, the audio path matters as much as the video path. By the end of this post, you should be able to diagnose the common causes of echo and audio artifacts, separate media issues from agent issues, and apply a few practical mitigations that actually hold up in production.

Start by classifying the symptom

“Echo” is often used as a catch-all, but there are several distinct failure modes:

• True acoustic echo: the speaker output is picked up by the microphone and sent back into the call.

• Digital feedback loop: the application routes the agent’s own playback back into the agent’s input stream.

• Double playback: the same audio is rendered twice, usually from two consumers attached to the same track or session.

• Artifacts from resampling: clipping, crackling, warbling, or robotic distortion caused by format conversion, jitter, or buffer underruns.

• Turn-taking glitches: the agent starts speaking before the user finishes, so the overlap feels like echo even though it’s actually barge-in or VAD behavior.

Treat these differently. Acoustic echo is a deployment/environment problem. Digital feedback is a wiring problem. Artifacts are usually codec, buffering, or sample-rate issues. Turn-taking issues are orchestration problems.

Trace the audio path end to end

The fastest way to debug a realtime avatar app is to draw the audio graph. You want to identify every hop:

1. Microphone capture from the browser, app, or telephony bridge.

2. Transport to your voice agent over WebRTC, WebSocket, or a vendor SDK.

3. ASR, LLM, and TTS stages inside the agent loop.

4. Playback to the user through the browser, client, or media server.

5. Optional avatar synchronization layer that consumes the same speech stream or timestamps.

Once you’ve mapped that, the main question is simple: can the playback path ever re-enter the input path? If yes, you need isolation. If no, you’re likely dealing with media quality or timing.

Echo is usually a routing problem, not a “model” problem

In browser-based apps, the most common acoustic echo bug is a speaker/microphone loop. The user hears the agent through the speakers, and the mic captures it again. In a call, that may be partially mitigated by the browser, OS, or device echo cancellation. In a desktop app or custom media pipeline, you may not get any help at all.

Practical checks:

• Use headphones first. If the issue disappears, you’re dealing with acoustic echo or a poor echo canceller.

• Mute the microphone while the agent is speaking. If the echo disappears, your input path is capturing playback.

• Temporarily disable local playback and inspect whether the agent still “hears itself.” If yes, you have a digital loop, not acoustic echo.

• Verify you are not subscribing to your own outbound track and feeding it back into ASR or the agent transport.

If your app uses a browser and you control the client, the usual mitigations are boring but effective: headphones, proper echo cancellation settings, and not playing the agent audio into the same capture graph that drives transcription.

Avoid duplicate consumers and loopback paths

Realtime avatar systems often have multiple consumers of the same speech stream: one for the user’s ears, one for lip sync, one for logging, one for analytics. That is fine as long as only one consumer is authorized to be “the speaker.” Problems start when the same audio frame is:

• played locally and also sent to another renderer,

• transcribed after it has already been mixed with output audio, or

• re-enqueued into the agent because the session bus does not distinguish inbound and outbound media.

This is especially easy to do in event-driven systems. A single “audio received” callback can accidentally subscribe to both the microphone stream and the avatar’s outbound stream if you are not strict about stream identity.

One useful rule: keep three streams conceptually separate at all times — user input, agent output, and rendered playback. They may share a transport, but they should never share semantics. If you cannot point at which one is authoritative at each hop, you probably have a loop waiting to happen.

Artifacts: sample rates, resampling, buffering, and clipping

Not all bad audio is echo. If the sound is warbly, phasey, crunchy, or intermittently robotic, inspect the media format first.

Common causes:

• Sample-rate mismatches: 48 kHz in one place, 16 kHz in another, with an uneven or low-quality resampler in between.

• Buffer underruns: the consumer cannot keep up with real-time playback, so audio starves and stutters.

• Excessive queueing: latency grows, then the app “catches up” with bursty playback that sounds broken.

• Clipping: voice gets too hot somewhere in the chain and peaks are flattened.

• Packet loss/jitter: especially in WebRTC and networked voice systems, where lost or delayed packets may be concealed poorly.

For developers, the right debugging move is to inspect the stream at each boundary. If your ASR wants 16 kHz mono PCM and your playback path is 48 kHz stereo, make the conversion explicit and put it in one place. Don’t let three libraries each “helpfully” resample the stream.

If you are using Python or a server-side agent loop, keep your audio pipeline deterministic. A small, explicit conversion step is preferable to letting every dependency make its own format assumptions.

from pathlib import Path

print("frames:", w.getnframes())

from pathlib import Path

print("frames:", w.getnframes())

from pathlib import Path

print("frames:", w.getnframes())

For live systems, you will not always have a WAV file to inspect. But the same discipline applies: log sample rate, channel count, frame size, and end-to-end latency at each boundary.

Turn-taking and barge-in can masquerade as echo

Sometimes the user says “it’s echoing,” but what they are really hearing is overlap. The agent starts speaking while the user is still talking, or the user interrupts the agent and the system does not properly cancel the current response. The result sounds like two voices stepping on each other.

This is usually controlled by voice activity detection, interruption handling, and playback cancellation. In practice, you want three behaviors:

1. Detect user speech quickly enough to avoid long overlaps.

2. Stop or duck playback when the user barges in.

3. Resume cleanly without replaying buffered audio after the interruption.

For avatar apps, this matters visually too. If the face keeps talking after the agent has been interrupted, users interpret the whole system as laggy or broken, even if the underlying audio transport is fine.

How Protoface fits into this debugging model

Protoface sits on the avatar side of the pipeline, so it is useful to think about it as a synchronized speech-to-face layer rather than a new audio transport. If you are integrating through the LiveKit Agents plugin, the main thing to verify is that the agent’s outbound speech stream is the one driving the avatar, and that you are not also routing that same stream back into transcription or playback.

The practical value here is that you can attach a talking face without rewriting your voice stack. If the audio is clean but the face looks wrong, the bug is in timing or session wiring. If the face and audio both glitch, the problem is earlier in the media path. The plugin repository and quickstarts are the right place to compare a known-good wiring pattern against your own setup; see the examples in the relevant GitHub repository and the broader docs at docs.protoface.com.

If you are creating or managing sessions programmatically, the REST API and Python SDK let you inspect the session lifecycle without involving the browser. That is handy when you need to verify whether a fault is in the client audio path or in the session orchestration itself. For example, a minimal API request might look like this:

curl -X POST https://api.protoface.com/v1/sessions \
-d '{"avatar_id":"avt_123","voice":"en-US"}'

curl -X POST https://api.protoface.com/v1/sessions \
-d '{"avatar_id":"avt_123","voice":"en-US"}'

curl -X POST https://api.protoface.com/v1/sessions \
-d '{"avatar_id":"avt_123","voice":"en-US"}'

Exact request fields depend on the endpoint and are documented in the API reference. The point is not the specific payload; it is that session creation should be reproducible enough that you can isolate media bugs from application bugs.

Debugging checklist that usually gets you to root cause

When a report comes in, I would work through this sequence:

• Test with headphones.

• Confirm whether the issue persists with local playback disabled.

• Verify the agent is not consuming its own outbound audio.

• Log sample rate, channels, and buffer sizes at each boundary.

• Check for duplicate subscriptions or repeated renderers.

• Test barge-in and interruption handling separately from echo cancellation.

• Compare against a known-good quickstart or sample integration.

If the issue only appears on one browser, one device class, or one network path, you are probably looking at environment-specific media behavior. If it appears everywhere, suspect your session graph or format conversion first.

Conclusion

Echo and audio artifacts in realtime avatar apps are usually the result of a small number of predictable problems: feedback loops, duplicate playback paths, sample-rate mismatches, buffering issues, or sloppy turn-taking. The right way to debug them is to trace the audio graph, separate input from output, and inspect media boundaries explicitly.

If you are integrating an avatar into a voice agent, keep the agent audio path and the avatar rendering path conceptually separate, then validate them one at a time. For implementation details, refer to the documentation at docs.protoface.com and compare your wiring against the quickstarts in the GitHub examples. That will usually get you from “it sounds broken” to a concrete root cause much faster than trying to tune the model.