End-to-End Testing a Realtime Avatar Stack with WebSocket, WebRTC, and LiveKit
End-to-end testing realtime avatar stacks with WebSocket, WebRTC, LiveKit: session setup, media flow, and lip-sync timing.
Introduction
End-to-end testing a realtime avatar stack is harder than testing a normal API or a normal video pipeline because you are validating three things at once: transport, media, and timing. A request can succeed at the HTTP layer, a WebSocket can stay open, and a WebRTC session can connect — and you can still end up with a dead avatar, desynced lip motion, or an agent that speaks without ever showing video.
This post focuses on how to test the whole stack as a developer would actually ship it: create a session, connect a realtime voice agent, attach a video face, and verify that the media path stays healthy under real network conditions. By the end, you should have a practical mental model for what to assert, where failures usually hide, and how to automate checks that catch regressions before users do.
What “end-to-end” means for a realtime avatar stack
For a typical avatar integration, the control plane and media plane are separate:
Control plane: create avatars, start sessions, configure voice/instructions, authenticate, and fetch session state.
Media plane: carry audio and video in realtime, usually over WebRTC, with signaling and coordination riding on a WebSocket or similar channel.
Your tests need to cover both. If you only check REST responses, you miss transport issues. If you only check that a WebRTC peer connection becomes connected, you may miss the fact that the avatar session was never created correctly or that the agent is not attached to the right face.
A useful end-to-end test usually answers four questions:
Can I authenticate and create the session I expect?
Does the agent attach to the avatar and establish media transport?
Does audio input produce timely, synchronized video output?
Does the system recover or fail cleanly when connectivity is imperfect?
Start with the control plane: create something you can verify
Before testing realtime behavior, make sure your session setup is deterministic and inspectable. The common failure mode here is “the test passed because the request returned 200, but the session configuration was wrong.” For that reason, your test should persist the returned identifiers and verify them against the session state after creation.
A minimal API flow usually looks like this:
In your test harness, do not hard-code assumptions about exact field names beyond what the docs specify. Instead, treat the returned session object as the source of truth and assert the properties you care about: avatar identity, voice configuration, lifecycle status, and any timestamps or URLs that the platform returns.
Using the Python SDK is often cleaner for integration tests than hand-rolling HTTP, especially when you want fixtures that create and tear down resources reliably:
If your tests run in CI, make teardown explicit. Realtime systems accumulate expensive resources quickly, and dangling sessions can distort usage and make later failures noisy.
Verify the media path, not just the signaling path
The easiest way to get false confidence is to treat “WebRTC connected” as success. In practice, the peer connection can be established while one direction of media is blocked, muted, delayed, or pointed at the wrong stream. Your test should observe actual media flow.
At a minimum, assert these conditions:
Signaling succeeds: WebSocket opens, SDP/offer-answer exchange completes, ICE reaches a usable state.
Audio flows: the agent receives microphone input and emits audio frames or a rendered audio track.
Video flows: the avatar video track becomes active and frames continue over time.
A/V relationship is plausible: the face moves in response to speech without large, sustained drift.
If you are testing from a browser or a headless browser, a practical strategy is to watch WebRTC stats rather than trying to “eyeball” pixels. Stats give you objective signals: bytes sent/received, packet loss, jitter, RTT, frames decoded, frames dropped, and current bitrate. Those metrics are enough to tell whether the session is alive and whether quality is degrading.
A lightweight test can also hash or sample video frames at intervals to confirm that the avatar is not frozen. You do not need computer vision sophistication to catch a dead stream; even checking that consecutive frames differ over time is a big improvement over checking only the connection state.
Test timing, because lip sync bugs are usually timing bugs
Realtime avatar stacks are sensitive to latency in ways that ordinary web apps are not. If audio arrives late, the avatar can still speak, but it will look wrong. If the video pipeline buffers too aggressively, the face lags behind the voice. If the test machine is overloaded, you can blame the network for what is really a local scheduling issue.
When designing tests, separate these concerns:
Transport latency: WebSocket and WebRTC setup time, ICE negotiation time, first media packet time.
Processing latency: time from user audio input to generated speech or video response.
Render latency: time until frames are actually visible in the client.
For regressions, the most important metric is usually “time to first meaningful media.” That is the point at which the avatar starts producing intelligible, synchronized output after the test begins. Track it over time and alert on large jumps.
Also test with realistic network behavior. WebRTC is designed to handle jitter and packet loss, but your application may still fail under conditions that look fine on localhost. Even a small amount of added latency can expose brittle assumptions about buffering and sequencing. If you can, run the same test in at least two environments: a local developer network and a CI or cloud runner with production-like constraints.
A practical browser-level test pattern
For avatar experiences embedded in a web app, a browser-level test is often the most representative end-to-end check. The sequence is:
Open the page and authenticate as the test user.
Request microphone permission or inject a test audio source.
Start the session.
Wait for WebRTC connection state to become usable.
Assert that video frames arrive and that the avatar responds to speech.
The key is to make the test observable. Log session IDs, connection states, ICE state transitions, and a few sampled stats. If the test fails, you want to know whether it died in session creation, signaling, media negotiation, or rendering.
A good debug trace is often more valuable than a screenshot. For example:
That kind of trace makes it obvious where a regression belongs. A frozen UI with healthy media stats is a frontend bug. Healthy signaling with no incoming frames is a transport or server-side media bug. A session that never reaches connected state is usually a signaling or auth issue.
Where Protoface fits in this stack
This is the part where Protoface helps: it gives you a consistent way to exercise the control plane and the voice-agent integration without having to hand-build every avatar session from scratch. In practice, that means your test can create or inspect sessions through the REST API or Python SDK, then drive a realtime agent through the LiveKit integration and validate that the synchronized talking face appears as expected.
If you are using a LiveKit voice agent, the plugin path is especially useful for integration tests because it keeps the avatar attachment close to the agent runtime rather than in a separate demo harness. The repo and installation notes are here: https://github.com/protoface-ai/protoface-plugin-pipecat for the Pipecat integration, and the broader docs are at https://docs.protoface.com. Use the docs for exact session fields, lifecycle semantics, and any quality-tier-specific behavior you need to account for.
One subtle advantage of testing through the real integration surface is that you are verifying the same synchronization path your users rely on. That matters more than a mock if your failure mode is “the avatar exists, but not when the agent is speaking.”
Common gotchas
Assuming localhost behavior matches production: WebRTC often looks perfect on a developer machine and fails under real latency or packet loss.
Trusting connection state too much: a connected peer connection does not guarantee usable audio or video.
Not isolating resource cleanup: orphaned sessions make later tests flaky and expensive.
Ignoring timing variance: realtime systems have natural jitter; set thresholds with enough margin to avoid false positives.
Over-mocking the wrong layer: mock business logic, not the transport path you are trying to validate.
Conclusion
End-to-end testing a realtime avatar stack is mostly about discipline: verify session creation, verify transport, verify actual media, and measure timing instead of guessing. If you do that, you will catch the failures that matter most — broken auth, bad session wiring, dead streams, and sync regressions — before they reach production.
If you are building on Protoface, start with the docs, then wire a small integration test around the surface you already use in production: REST, Python, or your LiveKit agent path. Keep the test short, observable, and repeatable. For implementation details and quickstarts, see docs.protoface.com and the examples linked from the project repositories.