Introduction

End-to-end tests for a realtime avatar app need to verify more than “the page loaded.” You are testing a chain of streaming systems: microphone input, audio transport, model/agent turn-taking, video frame generation, lip-sync timing, and browser playback. If any one of those drifts, the user experience breaks in ways that unit tests will never catch.

This post shows a practical way to add E2E coverage for a Vue 3 avatar client that receives streamed audio and renders a talking face. By the end, you should be able to build tests that:

• start a real session,

• send or simulate audio into the agent,

• assert that audio is received and playback starts,

• verify that the avatar video is updating while speech is active, and

• catch regressions in session setup, timing, and browser autoplay behavior.

I’ll assume you already have a Vue 3 app and a realtime avatar backend. The examples use Playwright because it handles browser-level assertions well, but the same structure applies to Cypress or Webdriver-based stacks.

What you should test, and what you should not

The first mistake teams make is trying to assert “lip sync correctness” at the pixel level. That is brittle and usually not necessary. The real contract is simpler:

1. when a session is active, the browser receives audio and video streams,

2. the avatar’s speaking state changes when speech starts and stops,

3. audio playback is aligned closely enough that the face appears to talk with the voice, and

4. reconnects, autoplay restrictions, and load timing do not silently fail.

That means your E2E tests should focus on observable behavior:

• WebRTC or streaming connection state transitions

• audio element ready/play events, or direct stream activity

• video frame updates over time

• UI state that reflects session health

Do not make tests depend on exact phonemes, frame-by-frame facial landmarks, or sample-perfect timing. Those belong in specialized media tests, not your app test suite.

Build a test harness around a real session

For end-to-end coverage, you want a test environment that uses the same session creation path as production, but with deterministic inputs. In practice, that means creating a disposable avatar session via your backend API, then attaching the browser to it.

If your application starts sessions through an API, keep the browser test thin: let the test create a session, load the Vue route, and wait for the client to attach.

import requests

print(session["id"], session["join_url"])

import requests

print(session["id"], session["join_url"])

import requests

print(session["id"], session["join_url"])

The exact request fields depend on your API shape, but the pattern matters: create a real session, then point the browser at the returned session identifier or join URL. If your app uses an embedded iframe instead, the same test principle applies: wait for the iframe to report readiness, then drive audio into the session and observe playback.

For a Vue 3 app, I usually expose explicit test hooks in the UI state so Playwright can tell whether the session is connecting, connected, speaking, or disconnected. That is much more stable than trying to infer everything from the DOM.

Verify audio streaming end to end

Audio is the first half of the contract. Your test should prove that the app can receive or send audio across the realtime boundary and that browser playback actually starts.

If the browser is the recipient, assert on the media element directly. A useful pattern is to wait for the audio element to exist, then confirm it has nonzero playback progress.

const audio = page.locator('audio[data-testid="avatar-audio"]');

});

const audio = page.locator('audio[data-testid="avatar-audio"]');

});

const audio = page.locator('audio[data-testid="avatar-audio"]');

});

That checks that the stream is live enough for playback. If your app uses Web Audio instead of a plain media element, assert on the underlying stream state or an application-level “audio-active” flag.

For the sending side, you have two realistic options:

1. Inject a prerecorded fixture audio file into the browser and let the app publish it into the realtime session.

2. Call your backend directly with a short test utterance and observe that the browser receives a response.

The first option exercises your client media pipeline. The second is better for validating the full agent loop, especially if the agent logic lives outside the browser.

Keep the fixture short and deterministic. Ten seconds of speech is usually enough. Use a stable sample rate and codec if your pipeline is sensitive to transcoding. If you can, record one clean mono file and reuse it across tests.

Verify lip-sync without overfitting

Lip-sync in a realtime avatar app is fundamentally a timing problem. The avatar video should track speech onset, sustain motion while audio is present, and relax when speech stops. You do not need a perfect biometric check; you need a regression detector.

A practical approach is to observe video activity during a speech window:

• the video element becomes ready,

• its frames advance while audio is playing, and

• the avatar’s speaking indicator, if available, toggles on during speech.

In Playwright, you can inspect whether the video element is producing frames by sampling its current time or by reading a canvas-fed frame counter from your app. If you render the avatar into a canvas, increment a counter on each frame and expose it as testable state.

await page.waitForFunction(() => {
});

await page.waitForFunction(() => {
});

await page.waitForFunction(() => {
});

That is not a lip-sync proof by itself, but combined with an audio assertion it tells you the session is actually streaming media. For tighter coverage, sample the UI state at a few points in time:

• before speech: avatar idle

• during speech: avatar speaking

• after speech: avatar returns to idle

Those transitions catch failures in turn detection, buffering, and “stuck speaking” bugs that users notice immediately.

A subtle but important gotcha: browser autoplay policies often block media playback unless the test has simulated a user gesture or the app handles the unlock flow properly. Make your E2E test click the “join” or “start” button as a real user would. If playback still fails, that is a real bug.

Make the test deterministic enough to be useful

Realtime systems are noisy. If your tests depend on exact wall-clock timing, they will flicker. A few rules help a lot:

• Use fixed-duration fixture audio.

• Wait on state transitions, not arbitrary sleeps.

• Set generous but bounded timeouts for session join and first frame.

• Keep the environment stable: same browser version, same network class, same media device setup.

I also recommend separating “smoke” coverage from “deep media” coverage. A smoke test can verify session creation, join, and one speaking cycle. A deeper media test can run less often and validate latency budgets or reconnect behavior.

One more thing: if your app uses a customer iframe embed, remember that the browser page and the avatar session have different security boundaries. Your test should confirm that the parent origin is allowed, the iframe loads, and the session is usable without exposing credentials in the browser. That is especially important when you are testing custom instructions or per-embed voice settings.

Where Protoface fits

If your avatar is backed by Protoface, you can keep the same E2E strategy and simply swap in the real session lifecycle. For voice agents built on LiveKit, the livekit-plugins-protoface plugin is the cleanest place to integrate: it drops a synchronized talking face into the agent, which makes it easy to test the full audio-to-video path end to end. For an iframe-based embed, your test can treat the iframe as the browser boundary and assert on the same playback and speaking-state signals.

For example, a lightweight Python test fixture can create a session through the API, then hand the resulting session metadata to your browser test:

from protoface import Client

print(session.id)

from protoface import Client

print(session.id)

from protoface import Client

print(session.id)

Keep the exact fields aligned with the docs; the important part is that your test uses real sessions rather than mocks. If you want a starting point, the public docs are the right source of truth: docs.protoface.com.

Conclusion

Good E2E tests for a realtime avatar app should prove media behavior, not just UI behavior. The most useful checks are: session creation succeeds, audio is actually played or published, video frames advance while speech is active, and the app recovers cleanly from the normal failure modes of realtime media.

Start with one smoke test that creates a real session and drives a short fixture audio clip through your Vue 3 client. Then add one or two focused assertions for speaking state and video frame progress. That gives you meaningful coverage without turning your test suite into a fragile media lab.

If you are wiring this into a Protoface-backed app, the docs are the best next step, and the LiveKit plugin or API flow you choose will determine where to hook your assertions. Once the harness is in place, the same pattern scales to voice agents, support bots, game NPCs, and any other realtime experience where the face has to keep up with the voice.