What Causes Audio Feedback in Realtime Avatar Calls? A Developer’s Guide to Fixing It
Diagnose audio feedback in realtime avatar calls: echo, digital loops, WebRTC track wiring, and fixes.
Introduction
Audio feedback in realtime avatar calls is almost never “just a microphone problem.” It usually means the audio graph has a loop: rendered audio gets picked up again by an input, reprocessed, and sent back out. In voice-agent and avatar systems, that loop can be obvious, or it can hide inside browser tabs, WebRTC tracks, SDK defaults, or a server-side mixer.
By the end of this post, you should be able to identify the common feedback paths, distinguish echo from true acoustic feedback, and apply the right fix in a browser, in a LiveKit-style agent stack, or in a custom backend.
What “feedback” actually means in a realtime avatar call
People use “feedback” to describe at least three different failure modes:
Acoustic feedback: a speaker emits audio, a microphone hears it, and the system re-sends it. This is the classic squeal/loop.
Echo: the remote end hears its own voice coming back delayed, usually because far-end audio is being captured locally or mixed into the outgoing track.
Digital loopback: audio is duplicated in software, for example when the same agent output is both played to the user and fed back into the agent input pipeline.
In avatar calls, digital loopback is especially common because the system often has multiple consumers for the same audio: the user hears it, the avatar lip-sync pipeline needs it, transcription may need it, and the agent may need to maintain conversational state. If those consumers are wired incorrectly, you can create a loop even when the physical mic and speakers are fine.
Where the loop usually comes from
The fastest way to debug feedback is to trace the audio path end to end. Think in terms of sources, transforms, and sinks:
Source: browser microphone, system audio, agent TTS output, or an incoming WebRTC track.
Transform: resampling, mixing, transcoding, VAD, transcription, or avatar video synthesis.
Sink: browser speakers, a WebRTC publisher, a recording pipeline, or a second agent.
Feedback appears when the output of one sink is also visible as a source somewhere upstream. In practice, that happens in a few recurring ways:
1. Browser mic captures playback. If you play agent audio through the speakers and the browser mic stays open, the mic may hear that playback. Hardware echo cancellation helps, but it is not perfect, especially with loud audio, open-air mics, headphones with poor isolation, or system-level loopback devices.
2. You publish the wrong track. In WebRTC stacks, it is easy to accidentally publish a mixed track that contains both user mic and remote audio, or to subscribe to your own outbound track and route it back into the recognizer.
3. The agent hears itself. If TTS output is inserted into the same stream that the speech recognizer uses as input, the model will eventually transcribe its own words and answer itself. This produces the “why is it talking over itself?” failure mode, which is not acoustic feedback but a pipeline loop.
4. Multiple tabs or sessions share a device. One tab plays audio; another tab captures the same microphone device or a virtual audio interface. The browser permissions look fine, but the operating system routing is wrong.
How to tell echo from a real loop
Before changing code, isolate the layer where the loop happens.
Mute the speaker output. If the problem disappears, it is probably acoustic or near-acoustic.
Unplug the microphone or disable capture. If playback continues but the agent stops re-responding, the issue is in the input path.
Log every audio edge. In a realtime pipeline, log when audio is received from the user, when it is sent to ASR, when TTS begins, and when avatar playback is attached. You want timestamps and track IDs, not just “started” and “stopped.”
Check whether the same samples are re-entering the system. If you can hash or fingerprint a short frame window, repeated fingerprints at regular delay intervals usually indicate a loop.
For WebRTC-based systems, browser echo cancellation and noise suppression can mask the issue in development and then fail in production when the user switches devices. That is why the correct fix is almost always architectural, not just “turn on AEC.”
Practical fixes that actually work
Keep the agent output out of the agent input. This is the main rule. If your voice agent needs to transcribe the user, feed it only the user’s capture track. If your avatar needs lip-sync audio, use a separate branch from the TTS output to the rendering component, not back into the recognizer.
Use half-duplex behavior when appropriate. If your interaction is simple and latency tolerance is high, stop listening while the agent is speaking. That eliminates the most common loop class. It is not ideal for truly interruptible conversations, but it is robust.
Prefer headphones for local testing. This sounds obvious, but it removes the physical path and makes software loops much easier to diagnose. If the issue only appears on speakers, you are probably looking at acoustic feedback, not a pure pipeline bug.
Apply VAD carefully. Voice activity detection should gate user input, not indiscriminately filter all audio. If you run VAD on mixed audio, it can treat the agent’s own speech as “user speech” and perpetuate the loop. Use separate channels or metadata to identify source.
Do not route playback through the same capture device. On desktops, virtual audio devices, “stereo mix,” and OS-level monitoring can create hidden loopbacks. In browser apps, avoid using system output as a capture source unless you explicitly need it.
Handle reconnects idempotently. Realtime calls often reconnect WebRTC transports or restart agent pipelines. If your reconnect logic attaches a new audio consumer without detaching the old one, you can get duplicated playback, which sounds like echo or comb filtering and is easy to misdiagnose.
Example: diagnosing a loop in a browser agent
A common failure pattern is: the user speaks, the agent replies, then the recognizer hears the reply and interprets it as a new turn.
The fix is not just “don’t do that” — it is to make the dataflow explicit. Treat user input, agent speech, and avatar playback as separate named streams. If you need a combined recording, create it as a downstream artifact, not as the live input to ASR.
Example: server-side guardrails in a realtime agent
On the backend, you can add a simple source check before forwarding audio into your recognizer or turn detector. The exact objects vary by stack, but the principle is stable.
This kind of guard is cheap and surprisingly effective. Even if your frontend misroutes a track during reconnect, the backend can prevent the worst-case loop from turning into a self-talking agent.
How Protoface fits in
If you are adding a synchronized talking face to an existing voice stack, the safest pattern is to keep the avatar attached to the agent’s speech output, not to the user input. That is exactly the shape supported by the LiveKit plugin and by the broader realtime session APIs in the docs. In other words: the avatar consumes the agent’s TTS/output stream for lip-sync and rendering, while the recognizer still only sees the human participant.
For LiveKit-based agents, the relevant integration surface is the plugin in the project’s examples and package docs on GitHub, which is the right place to look for track wiring and lifecycle details: quickstart examples. If you are building directly against the API, keep the avatar/session setup on the server and use your API key there; do not expose it in the browser. The browser-facing iframe embed is the exception when you want a customer-managed embed with no backend and no API key exposure on the client.
The operational benefit here is not magic feedback suppression. It is that the platform encourages a clean separation between the human input stream and the avatar output stream, which is the core discipline you need to avoid loops in the first place.
Debugging checklist
Confirm whether the symptom is acoustic feedback, echo, or a digital loop.
Trace which track is feeding ASR, which track is feeding TTS, and which track is feeding the avatar.
Verify reconnect logic does not attach duplicate consumers.
Test with headphones and then with speakers.
Keep agent output out of the recognizer path unless you explicitly want self-listening behavior, which you almost never do.
Conclusion
Realtime avatar feedback is usually a wiring problem, not an “AI” problem. The fix is to separate user capture from agent playback, make stream ownership explicit, and treat reconnects and mixed tracks as potential loop sources. If you do that, most feedback bugs become straightforward to reproduce and eliminate.
If you are building this into a production agent, start by reviewing the integration pattern in the docs, then test with a minimal audio graph before adding transcription, VAD, recording, or additional consumers. The fewer places the same audio can re-enter the pipeline, the fewer feedback bugs you will ship.