Introduction

If you are testing a realtime AI avatar interview agent, the hard part is not “does the model answer?” It is “does the whole loop stay coherent under latency, partial transcripts, interruptions, and media plumbing?” A voice interview agent usually combines speech-to-text, an LLM turn policy, and text-to-speech, then pushes the resulting audio and lip-sync events into a realtime video surface. That means your test strategy has to cover both application logic and the streaming contract between components.

This post shows a practical way to test that stack in Python with CI. The goal is to make your tests deterministic by mocking STT and TTS, then verifying the interview agent’s turn-taking, prompt handling, and avatar/session lifecycle without depending on live microphones, external model endpoints, or flaky timing. By the end, you should have a repeatable pattern for unit and integration tests that catch regressions before they hit a real session.

What makes realtime avatar agents different to test

Traditional chatbot tests are mostly request/response assertions. Realtime avatar agents are state machines with media edges:

• STT is streaming. You may receive partial hypotheses, final transcripts, timestamps, or utterance boundaries. Your agent often needs to react only on finalization.

• TTS is streamed. Audio typically starts before the full sentence is synthesized, and cancellation matters when the user interrupts.

• The avatar is synchronized to audio. If the audio stream stalls, the face should not keep “talking” forever. If the turn changes, the speaking state should reset promptly.

• Timing is nondeterministic. Network jitter, scheduler variance, and async callbacks can make naive tests flaky.

That is why you want to isolate the interview logic from the media providers. Treat STT and TTS as interfaces, then feed your agent controlled events in tests. Your assertions should focus on observable behavior: what prompt was generated, when the agent started or stopped speaking, whether interruption canceled the prior response, and whether session metadata was created correctly.

Design the agent so it can be tested

The most useful change you can make is architectural: keep the interview policy in a plain Python class that depends on abstractions, not on live SDK objects. For example:

• Transcript source: yields partial and final user utterances.

• LLM layer: turns transcript state into a reply.

• TTS layer: turns text into audio chunks or a stream.

• Avatar output: consumes audio and speaking-state events.

That lets you test the logic with small fakes. The production adapters can wrap your actual realtime stack, but the core interview policy stays deterministic.

from dataclasses import dataclass

return reply

from dataclasses import dataclass

return reply

from dataclasses import dataclass

return reply

This is intentionally boring code. Boring is good: it is easy to mock, easy to observe, and easy to reason about in CI.

Mock STT and TTS in tests

For unit tests, do not use real ASR or synthesis. Instead, simulate the event sequence your production system would see. That means feeding partial and final transcript events into the agent, and returning predictable audio chunks from the TTS mock.

import asyncio

assert agent.avatar.events == ["start", b"audio-1", b"audio-2", "stop"]

import asyncio

assert agent.avatar.events == ["start", b"audio-1", b"audio-2", "stop"]

import asyncio

assert agent.avatar.events == ["start", b"audio-1", b"audio-2", "stop"]

This kind of test catches the basic contract: only final transcripts trigger a response, and speech lifecycle events wrap the audio stream exactly once.

Test interruption and turn-taking explicitly

Realtime interview agents break most often when the user interrupts mid-answer. The fix is not just “handle cancellation”; you should test the specific turn boundary you care about. For example, when a new final transcript arrives while the avatar is still speaking, your policy might cancel the in-flight TTS stream and start a new turn.

In tests, model that as a controlled async generator and a cancellation path. You do not need real timing; you need to assert that your code calls the right cancellation hook in the right order.

class CancelableTTS:

self.cancelled = True

class CancelableTTS:

self.cancelled = True

class CancelableTTS:

self.cancelled = True

Then write a test that injects a second transcript and verifies the previous turn is canceled before a new reply is started. The exact mechanics depend on your agent framework, but the assertion should always be about behavior, not implementation details:

• the old TTS stream is canceled,

• the avatar stops speaking before the next answer starts,

• the new user utterance becomes the active turn.

A useful extra check is transcript deduplication. Some STT systems emit repeated partials or corrected finals. Your agent should avoid answering twice when the same final text is delivered multiple times.

Run integration tests in CI without real STT/TTS

Once the unit tests are in place, add one lightweight integration test that exercises the concrete adapter layer, but still keeps STT and TTS mocked. In CI, the goal is not media fidelity; it is catching regressions in session setup, auth, event wiring, and serialization.

Common practices that keep these tests stable:

• Pin async timeouts. Use short, explicit timeouts around awaited events so deadlocks fail fast.

• Record the event order. Your assertions should check sequence, not just presence.

• Avoid sleep-based synchronization. Prefer queues, futures, or callbacks you can await directly.

• Separate network tests from logic tests. If a test needs real API credentials, tag it as a separate job and do not run it on every push.

import asyncio

assert events[-1] == "stop"

import asyncio

assert events[-1] == "stop"

import asyncio

assert events[-1] == "stop"

In GitHub Actions or any other CI runner, this pattern stays fast because there is no dependency on microphone input, browser automation, or external TTS latency. If you need a smoke test against the real service, keep it separate and gated on secrets.

Where Protoface fits: test the avatar/session boundary, not the media providers

This is where Protoface is useful. In a production deployment, your interview agent may already have a live voice stack, and Protoface adds the synchronized avatar/session layer on top of it. For testing, the right move is usually to mock STT and TTS in Python, then validate that your agent integrates cleanly with the avatar surface you use in production.

If you are using the LiveKit Agents plugin, the plugin itself can be exercised in a thin integration test while the underlying STT/TTS remain faked. The idea is to verify that your avatar receives the correct speaking-state transitions and that session wiring is correct. The same pattern applies if you create sessions through the REST API or SDK: make a small number of assertions around session creation and lifecycle, not around generated speech content.

# Illustrative only: exact fields and methods are in the docs.

assert session.id

# Illustrative only: exact fields and methods are in the docs.

assert session.id

# Illustrative only: exact fields and methods are in the docs.

assert session.id

For the LiveKit path, the plugin is published on PyPI as livekit-plugins-protoface; if you are integrating through LiveKit, the examples in the plugin repo are the right place to mirror your production wiring. The important testing point is that the avatar layer should be exercised as an adapter around a deterministic interview policy, not as the place where business logic lives. If you want a reference for the Python SDK, the repository at github.com/protoface-ai/protoface-sdk-python is the most relevant starting point. The public docs at docs.protoface.com cover the exact request shapes and session fields.

CI checklist and common gotchas

Before you call the test suite done, check the failure modes that usually slip through:

• Final vs partial transcripts. Make sure partial STT events do not trigger answers.

• Cancellation races. If a new user turn arrives during TTS, verify the old stream cannot continue pushing audio.

• Speaking-state drift. The avatar should always return to idle after the last audio chunk.

• Prompt contamination. Each turn should start from the intended conversation state, not stale buffer contents.

• Auth and config. Add one test that validates API key plumbing and environment-variable loading, but keep secrets out of logs.

If you have an interview-specific rubric, this is also where you encode it. For example, you can assert that the first question is concise, that follow-ups reference the candidate’s last answer, or that a timeout path produces a graceful prompt instead of a hung session. Those are logic-level requirements, and they belong in deterministic tests.

Conclusion

The practical way to test a realtime AI avatar interview agent in Python is to keep the media stack thin and the policy layer testable. Mock STT and TTS, feed controlled transcript events, assert on turn-taking and cancellation, and run the whole thing in CI with short timeouts and no external dependencies. Then add a small number of adapter tests for the avatar/session boundary so you know the production wiring still works.

If you are integrating a realtime avatar into a voice agent, start with the docs at docs.protoface.com, and keep the code under test focused on the behavior you actually care about: when the agent speaks, when it stops, and how it responds to interruption.