Introduction

When you add a talking avatar to a product, the failure modes multiply. You are no longer testing only text generation or audio playback; you are testing a chain that typically includes an LLM, a TTS service, a streaming transport, lip-sync timing, and a video surface. If your end-to-end tests depend on a live TTS vendor every run, they become slow, flaky, and expensive very quickly.

This post shows a practical way to mock TTS in end-to-end avatar tests without turning the test into a meaningless unit test. The goal is to keep the avatar pipeline realistic enough to validate orchestration, timing, and rendering while removing the external dependency that most often causes instability. By the end, you should be able to design tests that are deterministic, fast, and still catch the kinds of regressions that matter in realtime avatar systems.

What you actually want to test

In a realtime avatar stack, “TTS” is usually just one stage in a larger stream. The important behavior is not whether a vendor synthesized a perfect voice; it is whether your system correctly handles the boundaries between text, audio chunks, and video updates.

For end-to-end tests, the useful questions are:

• Does the agent forward generated text into the speech layer at the right time?

• Does the avatar session start, stay alive, and clean up correctly?

• Do audio chunks arrive in a format the avatar pipeline can consume?

• Does lip-sync or mouth motion continue smoothly when the stream is segmented?

• Do retries, cancellations, and timeouts behave as expected?

If you mock too aggressively, you only test that your code can call your own mocks. If you mock too little, the test becomes hostage to a third-party API. The useful middle ground is to preserve the protocol shape and timing profile while substituting a deterministic local source for the real TTS provider.

Mock the boundary, not the whole pipeline

The right seam is usually between your agent logic and the TTS provider. Keep the avatar runtime, transport, and session management as real as possible. Replace only the external text-to-speech dependency with a local stub that emits predictable audio events or audio files.

There are three common patterns:

1. Static audio fixture: return a known WAV/MP3 file for any input. Good for basic smoke tests.

2. Deterministic synthesis stub: generate simple tone bursts or concatenated phoneme-like segments based on text length. Good for timing-sensitive tests.

3. Recorded response replay: capture real TTS responses once, then replay them in CI. Good when you need realistic audio characteristics without network calls.

For avatar systems, the second option is often the best compromise. It preserves streaming behavior and makes the test sensitive to chunking and backpressure, which is where many bugs hide.

Designing a useful TTS mock

A practical mock should behave like a real streaming TTS service in the ways your code depends on. That usually means:

• accepting text input asynchronously

• emitting audio in chunks rather than a single blob

• supporting cancellation when the user interrupts the agent

• optionally injecting latency so you can observe buffering behavior

Keep the implementation simple. A mock does not need natural speech. It needs stable output and a believable control flow.

import asyncio

yield chunk

import asyncio

yield chunk

import asyncio

yield chunk

If your downstream code expects WAV frames, generate a small deterministic file instead of arbitrary bytes. The exact codec and container should match what your avatar pipeline consumes. In tests, consistency matters more than realism.

Two additional details matter a lot in practice:

• Stable timestamps: if your avatar pipeline uses timestamps to drive mouth motion, your mock should preserve a fixed cadence across runs.

• Interruptibility: if the user speaks over the agent, the mock must stop cleanly. A test that never exercises cancellation is incomplete.

Keep the realtime transport real

The main mistake people make is mocking the entire realtime stack because they only wanted to avoid one external API. That removes the very behavior you need to validate: session lifecycle, stream buffering, and synchronization.

For voice agents and avatars, I prefer this test shape:

1. Start a real session or local integration harness.

2. Inject a mock TTS adapter at the edge of the agent.

3. Feed a known user utterance into the agent.

4. Assert that audio/video activity appears within a timeout.

5. Assert that the session closes cleanly on completion or interruption.

This gives you confidence that the entire realtime path works without depending on the external TTS provider’s availability.

import asyncio

await agent.stop()

import asyncio

await agent.stop()

import asyncio

await agent.stop()

If you are testing browser behavior, the same principle applies: run the real browser, the real iframe or websocket transport, and a fake TTS backend. Do not replace the browser with a mock unless the test is explicitly about browser logic.

Common failure modes and how to avoid them

A good mock should help surface bugs, not hide them. These are the failures I see most often:

• Tests only validate the happy path: add cancellation and timeout cases, not just successful synthesis.

• Mocks ignore chunk boundaries: many bugs only appear when audio arrives in multiple pieces.

• Fixtures are too small: a one-frame sample can miss buffering and end-of-stream issues.

• Timing is completely unrealistic: if the mock returns instantly, you will not catch race conditions in your avatar pipeline.

• Assertions are too vague: assert on session state transitions, emitted events, and cleanup, not just “no exception raised.”

When in doubt, bias toward determinism. Real-time systems are already noisy enough. Your test harness should remove randomness, not add more.

How Protoface fits into this

This is exactly the kind of integration problem Protoface is built around: a developer-facing realtime avatar API that you can drive from a voice agent, a backend session controller, or an embedded web experience. For test purposes, the key point is that you can keep the avatar/session layer real while swapping out the TTS dependency underneath your agent.

If you are wiring a Python voice agent, the Python SDK and the LiveKit plugin approach both make this boundary fairly natural: your agent still thinks it is speaking to an avatar service, but in tests you can inject a fake TTS provider before the audio ever leaves your process. That keeps the avatar lifecycle and transport realistic without burning time or money on live synthesis calls.

For lower-level integration tests, the REST API is useful when you want to create or manage sessions programmatically. The exact request/response fields are documented in the docs, but the test strategy stays the same: let the avatar platform do the real session work, and mock only the external speech source.

Example test strategy in practice

A good CI setup usually has three layers:

1. Unit tests for prompt logic, interruption handling, and adapter code.

2. Integration tests with mocked TTS for realtime avatar/session behavior.

3. Occasional contract tests against real TTS to catch provider-specific schema or codec changes.

That last layer should be small and intentionally non-blocking if the vendor is flaky. You only need enough coverage to detect when the real provider changes behavior in a way your mock would not catch.

If you are using a browser embed, the same pattern still applies: keep the iframe and session flow real, but drive the spoken content from a controlled backend fixture rather than a live TTS call. That lets you validate parent-origin restrictions, session startup, and teardown without exposing API keys in the browser or introducing an external network dependency into the test.

Conclusion

Mocking TTS in end-to-end avatar tests is not about faking the entire product. It is about choosing the right seam so that your tests still exercise the realtime behavior that matters: streaming, timing, cancellation, and cleanup. Keep the avatar/session layer real, make the TTS boundary deterministic, and assert on state transitions rather than just output bytes.

If you are implementing this in a Protoface-based stack, start by isolating the TTS adapter in your agent and then run your avatar integration tests against the real session path. The docs at docs.protoface.com are the best place to check the exact SDK and API shapes, and the quickstart repos linked from the project are useful references when you need a working baseline.