Introduction

If you’re building a realtime AI avatar customer-support bot, the hardest part to test is not the LLM prompt or the REST endpoint. It’s the full loop: microphone audio in, WebRTC transport up, ASR/agent/TTS latency under control, avatar video synchronized with speech, and the browser UI behaving like a real customer session.

That’s the kind of system where “works on my machine” is meaningless. You need an end-to-end test that exercises the same browser path your users take, verifies that media actually flows, and checks the bot’s observable behavior rather than just HTTP responses.

In this post, I’ll show how to structure an E2E test for a realtime avatar support bot using Playwright on the browser side and WebRTC-aware assertions on the media side. By the end, you should be able to automate a realistic customer interaction, confirm that audio/video setup succeeds, and assert that the avatar responds within the latency budget you care about.

What you’re really testing in a realtime avatar flow

A support bot with an animated face is a distributed system. A useful test has to cover at least four layers:

• Browser transport: the page loads, permissions prompts are handled, and the WebRTC peer connection reaches a connected state.

• Realtime media: microphone audio is captured and sent; remote audio and avatar video are received and rendered.

• Agent behavior: the backend hears the user, produces a response, and emits speech/video in the correct order.

• Product behavior: the bot answers a support scenario correctly, not just syntactically.

Do not try to verify this with a single DOM assertion like “the chat bubble contains text.” That misses the important regressions: broken SDP negotiation, muted tracks, frozen video, laggy turn-taking, or an avatar that is technically connected but never visibly updates.

Design the test around observable media events

For a browser-driven test, Playwright should do what a user does: open the support page, allow mic/camera permissions if needed, join the session, speak or inject audio, then observe the avatar and transcript. The key is to make media state observable from the page, not from private implementation details.

In practice, I look for three checkpoints:

1. Connection checkpoint: the page reports that the session is live and the WebRTC peer connection is connected.

2. Turn-taking checkpoint: after user input, the bot produces a response within a reasonable timeout.

3. Avatar checkpoint: the remote video element is playing and its frames advance during the response.

If your app exposes session state in the DOM, test that. If it exposes a transcript panel, assert against that. If it shows a video element, make sure it’s actually receiving a stream and not just rendering a poster frame.

Minimal Playwright pattern for a realtime session

The exact UI depends on your app, but the test skeleton is usually the same. Here’s a compact example:

import { test, expect } from '@playwright/test';

});

import { test, expect } from '@playwright/test';

});

import { test, expect } from '@playwright/test';

});

This is intentionally not enough on its own. A visible video element doesn’t prove frames are changing. For that, add a small page-side helper that inspects the media element state or samples frames with Canvas if your app allows it.

Checking WebRTC health without overfitting to internals

WebRTC failures are often subtle. The peer connection can be “connected” while the inbound video track is dead, or audio can be flowing while the avatar freezes because rendering stalled. Don’t rely only on console logs; expose a few metrics from the page if you can.

A practical pattern is to publish a diagnostic object from the client app in test mode:

window.__rtcDebug = {
};

window.__rtcDebug = {
};

window.__rtcDebug = {
};

Then in Playwright you can poll that state:

await expect.poll(async () => {
}).toBe(true);

await expect.poll(async () => {
}).toBe(true);

await expect.poll(async () => {
}).toBe(true);

If you want a stronger signal that frames are advancing, attach a canvas to the video element in test mode and sample a few pixels over time. That’s more work, but it catches the common “stream connected, frame stuck” failure.

Make the user interaction deterministic

Realtime tests get flaky when they depend on natural speech timing or on random agent output. Stabilize the interaction path:

• Use a fixed prompt or scripted utterance.

• Keep the environment quiet; do not run these tests on noisy CI machines if you can avoid it.

• Prefer short, unambiguous support prompts such as “I was charged twice” or “Where is my invoice?”

• Assert on intent or structured state, not exact prose.

If your app supports text input that the backend turns into speech, use that for the E2E assertion path and keep a separate media test for the audio pipeline. If you need to test live microphone capture, treat it as a dedicated integration test class, because audio-device handling is one of the most environment-sensitive parts of the stack.

What to validate in the bot response

The best support-bot tests check behavior, not style. For example:

• Did the bot acknowledge the issue?

• Did it ask for the right follow-up question?

• Did it avoid unsupported claims?

• Did it keep the response under the target latency?

You can implement this with a simple set of assertions over the transcript or over the last assistant turn. A common failure mode is the agent producing a semantically correct answer but taking too long to start speaking, which feels broken to users even if the final text is fine.

For timing, instrument the client with timestamps for:

• user action sent

• first assistant token or first transcript update

• first audible output or video motion

That gives you a meaningful “time to first response” number, which is usually more actionable than end-to-end completion time.

Where Protoface fits in this setup

This is the part where Protoface matters: it gives your voice agent a synchronized talking face, so your test can validate the avatar path instead of treating it as an afterthought. If you’re using the LiveKit stack, the livekit-plugins-protoface plugin is the cleanest way to drop an avatar into an existing agent and keep video aligned with speech. If you need to provision avatars or sessions from code, the REST API and Python SDK are the natural control plane.

A minimal provisioning call looks like this:

curl -X POST https://api.protoface.com/v1/sessions \

curl -X POST https://api.protoface.com/v1/sessions \

curl -X POST https://api.protoface.com/v1/sessions \

Exact request fields depend on the flow you’re using, so treat this as illustrative and check the docs for the current schema. The useful part from a test perspective is that you can create a session, launch your browser against it, and assert on the resulting media behavior in a fully automated run.

If you’re wiring the avatar directly into a LiveKit voice agent, the integration shape is also simple. The plugin sits in the agent pipeline, so when the agent speaks, the avatar renders synchronized video output rather than an unrelated webcam or static image. That’s exactly the behavior your E2E test should validate.

Common gotchas and how to avoid them

A few issues come up repeatedly:

• Permissions prompts: grant mic/camera permissions in the browser context; do not click through system dialogs manually in CI.

• Autoplay restrictions: remote audio may require a user gesture before playback. Make the test click a real button first.

• Timing flakiness: use generous but bounded timeouts for the first test, then tighten them once you know your latency envelope.

• Over-asserting transcript text: small model changes will break brittle text matches. Prefer semantic checks.

• Testing the wrong layer: if you only test REST endpoints, you won’t catch WebRTC regressions. If you only test the browser UI, you won’t know whether the backend agent actually handled the turn.

In CI, keep one “happy path” E2E test and a few targeted failure tests. For example, verify that reconnect logic works, or that the session fails cleanly when credentials are missing. Don’t try to simulate every edge case in one giant browser script.

Putting it together in a practical workflow

A good workflow looks like this:

1. Provision a session or avatar in setup.

2. Launch a Playwright browser with media permissions enabled.

3. Join the realtime support page and wait for connected state.

4. Send a scripted customer issue.

5. Assert on transcript semantics, response latency, and avatar playback.

6. Capture artifacts: console logs, screenshots, and any app-level RTC diagnostics.

That gives you enough signal to debug failures quickly. When a test breaks, you can usually tell whether the problem is the page, the transport, the agent, or the avatar rendering path.

Conclusion

For a realtime AI avatar customer-support bot, end-to-end testing is about validating the whole media and conversation loop, not just a web page. Use Playwright to drive the browser like a user, expose a small set of RTC diagnostics, and assert on observable behavior: connection state, response timing, transcript semantics, and avatar playback.

If you’re integrating an avatar into a voice agent or creating sessions programmatically, check the docs at docs.protoface.com. If you want a concrete starting point, the quickstarts linked from the repository are a good way to adapt this pattern to your stack. Once you have one reliable happy-path test, expand from there with reconnect, timeout, and permission-denied cases.