What Is the Difference Between Echo Cancellation and Noise Reduction in STT Pipelines?
Echo cancellation vs. noise reduction in STT pipelines: how they differ, where each fits, and when to use both.
Introduction
In a speech-to-text (STT) pipeline, two different classes of audio cleanup often get lumped together: echo cancellation and noise reduction. They are related, but they solve different problems, operate on different signals, and fail in different ways. If you’re building a voice agent, a realtime avatar, or any interactive audio product, mixing them up usually leads to avoidable latency, duplicated processing, or degraded transcription quality.
By the end of this post, you should be able to distinguish the two, understand where each fits in a realtime pipeline, and make a sane decision about what to enable upstream of STT versus what to leave to the ASR model itself.
Echo cancellation: removing the system’s own audio from the microphone
Echo cancellation exists because the microphone often hears not just the user, but also audio coming from the device’s speakers. In a conversational app, that “speaker bleed” may be your assistant’s synthesized voice, a sound effect, or any remote participant’s audio being played locally. If you send that mixed signal into STT, the model may transcribe its own output, the other speaker’s output, or both.
Acoustic echo cancellation (AEC) is specifically designed to estimate the playback signal and subtract the portion of it that leaks back into the microphone. In practice, AEC needs two inputs:
Near-end mic signal: what the microphone actually captured.
Far-end reference signal: the audio being rendered to the speakers, time-aligned as closely as possible.
The “reference” requirement is the key distinction. Without it, you can still do general denoising, but you cannot reliably remove echoed playback because the algorithm doesn’t know what to subtract.
On realtime systems, AEC is usually handled by the browser, OS, WebRTC stack, or a low-level audio library. That’s because proper echo suppression depends on tight timing, device characteristics, and access to both playback and capture streams. If you’re running a voice agent in a browser and you let the assistant talk through the same speakers that the microphone is listening to, AEC is often mandatory.
Noise reduction: improving signal quality without knowing the source
Noise reduction is broader and more generic. It tries to make speech more intelligible by reducing background noise: fan hum, keyboard clicks, HVAC, traffic, room tone, transient bumps, and similar artifacts. Unlike echo cancellation, noise reduction does not need a playback reference. It only sees the microphone signal and tries to infer which components are likely speech versus noise.
This makes noise reduction useful in a much wider set of situations, but also inherently less precise. A denoiser can reduce steady-state noise, but it can also distort speech harmonics, clip consonants, or introduce musical artifacts if tuned aggressively. Modern STT models are often fairly robust to moderate noise, so “more denoising” is not always better.
Think of noise reduction as a signal-quality improvement step, not a semantic subtraction step. It can help ASR, but it can also harm it if you overprocess the input.
Where they sit in a realtime STT pipeline
A practical realtime voice pipeline usually looks something like this:
That order is not absolute, but it reflects the usual dependencies:
Echo cancellation first, because it removes audio that originated from the app itself or the remote peer and is most likely to confuse the recognizer.
Noise reduction next, because once you have a cleaner speech signal, general denoising can help more than it hurts.
Voice activity detection (VAD) after cleanup, because both echo and noise can create false positives.
STT on the cleaned stream, ideally with low added latency.
There are two important caveats:
Do not assume AEC is a substitute for denoising. Echo cancellation removes a known playback signal; it does not address background noise.
Do not assume denoising can fix echo. Echo is correlated with a local playback reference, not just generic noise. A denoiser may partially attenuate it, but often not enough, and sometimes it will make it worse by smearing the reflected speech into the mic signal.
Failure modes developers actually run into
Most debugging pain comes from incorrect assumptions about what the audio pipeline is already doing.
1. “My bot hears itself.” This is usually a missing or ineffective AEC problem. The assistant speaks out of the speakers, the mic captures that output, and the STT layer transcribes the assistant’s own words as if they were the user’s. You may also see barge-in logic fire on the bot’s voice, which creates a feedback loop.
2. “Transcripts are clean in silence but bad in noisy rooms.” That’s a noise reduction problem, not an echo problem. The fix may be better microphone placement, less aggressive denoising, or a model that tolerates noise better.
3. “Turning on all the cleanup options made things worse.” This is common when browser AEC, SDK-level denoising, and server-side preprocessing all run at once. Cascaded processing can create latency, phase distortion, and artifacts that confuse the STT model more than the original noise would have.
4. “The model hears clipping or choppiness.” Heavy noise suppression can gate speech segments too aggressively, especially on low-energy consonants and trailing syllables. Your STT may then miss words that were actually present in the raw audio.
If you’re troubleshooting, isolate each stage. Start with raw mic capture, inspect whether echo is present, and then enable one cleanup component at a time. For realtime systems, latency and artifact profile matter as much as word error rate.
How to choose the right tool in practice
A simple rule of thumb works well:
Use echo cancellation when your app plays audio locally while simultaneously listening to the user.
Use noise reduction when the capture environment is noisy or inconsistent.
Use both when you have a full-duplex conversational system.
For browser-based apps, the browser and WebRTC stack already expose AEC-related controls in many cases. For server-side pipelines, the burden shifts to your audio SDK or media infrastructure. The quality of the implementation matters: timing alignment for AEC is fragile, and generic denoising varies a lot in aggressiveness.
Also, remember that STT models themselves increasingly incorporate robustness to noise, overlap, and mild reverberation. If your input is already reasonably clean, pushing more preprocessing may not improve accuracy. Sometimes the best optimization is simply to avoid degrading the audio before it reaches the recognizer.
How Protoface fits in
When you add a realtime avatar into a voice agent, this distinction becomes more than academic. The avatar is producing the assistant’s audio, and your microphone path is capturing the human user. If the playback audio leaks into the capture stream, the STT layer can transcribe the agent’s own speech unless AEC is handled correctly somewhere in the stack.
The Protoface LiveKit integration is a good example of where this matters in practice. If you are using the LiveKit Agents plugin, the avatar becomes part of a full-duplex conversational loop, so your audio pipeline should already account for speaker playback versus microphone capture. The plugin’s examples are a useful reference for wiring the avatar into an existing voice agent without treating echo and noise as the same problem; see the OpenAI Realtime quickstart and the project docs at docs.protoface.com for the integration shape and current fields.
The key point is architectural: if the avatar is speaking through the same output path that the user’s microphone can hear, you need reliable echo cancellation somewhere before STT. If the environment is also noisy, then add noise reduction as a separate, measured step rather than assuming one knob handles both.
Conclusion
Echo cancellation and noise reduction are not interchangeable. Echo cancellation removes known playback leakage from the microphone path; noise reduction suppresses unknown background noise. In a realtime STT pipeline, AEC usually belongs before general denoising, and both should be evaluated against latency and artifact risk, not just perceived “cleanliness.”
When you build conversational voice products, especially with realtime avatars, make the audio path explicit and test it under realistic conditions: speakers on, mic near a keyboard, bad room acoustics, overlapping speech, and barge-in. That is where the differences show up.
If you need a concrete integration target, start from the relevant examples in the docs or the GitHub quickstarts, then measure what your STT actually receives before and after each preprocessing step. That will usually tell you more than any generic recommendation.