Introduction

When you add voice and video to a customer support agent, you stop testing “an API call” and start testing a realtime system. The load profile changes: each session holds open a transport, emits and receives low-latency media, and often fans out into STT, LLM, TTS, and avatar rendering. That means the bottlenecks are no longer just request rate and CPU. They are connection churn, backpressure, media timing, session orchestration, and tail latency under concurrency.

This post shows a practical way to load test that kind of stack with FastAPI, WebSocket clients, and Locust. By the end, you should be able to:

• stand up a small FastAPI service that behaves like a realtime support agent endpoint,

• drive many concurrent websocket sessions with Locust,

• measure the metrics that matter for voice/video agents, and

• understand where a realtime avatar layer fits without distorting your test results.

Model the system you actually run

The first mistake people make is benchmarking a single HTTP endpoint and calling it “voice agent performance.” A real customer-support call has at least four moving parts:

• Session setup: auth, conversation state, and media negotiation.

• Realtime transport: usually WebSocket for control/data and WebRTC or a streamed media path for audio/video.

• Agent turn loop: user audio in, STT, LLM response, TTS out.

• Rendering path: for video avatars, the synthesized face must stay synchronized with the agent’s speech.

For load testing, you do not need the full production media stack on day one. You need a representative contract. A good test harness simulates:

1. session creation,

2. an open websocket per user,

3. a stream of user “turns” or audio chunks,

4. the server’s response timing, and

5. session teardown and cleanup.

That gives you latency, concurrency, and failure behavior. If you later want to test media codecs or a browser iframe, you can layer that on, but start with the simplest transport that still exercises your application logic.

Build a minimal FastAPI websocket target

FastAPI is a convenient way to create a controllable target service for load tests. The point is not to mimic your full production agent, but to create a stable, observable endpoint with similar connection behavior.

from fastapi import FastAPI, WebSocket, WebSocketDisconnect

return

from fastapi import FastAPI, WebSocket, WebSocketDisconnect

return

from fastapi import FastAPI, WebSocket, WebSocketDisconnect

return

A few details matter here:

• Keep the websocket open for the full conversation. Connection setup costs are real, but so is steady-state behavior.

• Return a small structured payload so your client can measure round-trip time and verify ordering.

• Make “think time” configurable per message; otherwise every virtual user produces identical traffic, which hides queueing behavior.

If your production agent processes audio rather than text, replace receive_text() with binary frames or a control protocol that reflects your actual ingest path. For load testing, the payload does not need to be real speech unless you are explicitly benchmarking STT or transcoding.

Load test websocket concurrency with Locust

Locust is useful here because it lets you define a virtual user that holds a websocket open, performs a sequence of actions, and records timings per session. The key is to model user behavior, not just socket count.

import json

print(f"turn_latency_ms={latency_ms:.1f}")

import json

print(f"turn_latency_ms={latency_ms:.1f}")

import json

print(f"turn_latency_ms={latency_ms:.1f}")

That example uses a blocking websocket client for clarity. In practice, the client library you choose matters less than the behavior you simulate. The important part is that each Locust user keeps a live connection and exercises the same sequence your production agent uses.

What to measure, and what not to trust

For realtime agent systems, the wrong metrics are seductive. Raw requests per second only tells you how many messages you can shove through the system. It does not tell you whether the conversation feels usable.

Track these instead:

• Session setup latency: time from connect to first usable response.

• Turn latency: time from user message/audio chunk to assistant delta.

• Tail latency: p95 and p99, not just averages.

• Open connection count: websockets hold file descriptors and memory.

• Failure rate: disconnects, timeouts, malformed frames, and queue overflows.

• Server saturation signals: event loop lag, worker CPU, memory growth, and downstream provider throttling.

There are a few common gotchas:

• Warm caches hide startup cost. Test both cold start and steady state.

• Short runs lie. Realtime systems often degrade after enough connections accumulate.

• Uniform traffic is unrealistic. Mix short support questions with longer “escalation” turns.

• Client-side timing can be noisy. Keep the load generator on a stable machine and monitor its own CPU and socket limits.

If your production architecture uses streaming audio, be careful not to overfit to your load harness. Sending one message every 150 ms is not the same as streaming 20 ms PCM frames. If your bottleneck is jitter buffers, codec overhead, or packetization, you will need a media-aware test later. But if your bottleneck is agent orchestration or backend queuing, the websocket harness above will get you most of the way there.

Bring in a realtime avatar layer without changing the testing model

If the support agent also renders a talking face, the relevant test question is not “can the avatar produce beautiful video under load?” It is “does the avatar stay synchronized with the agent under realistic session concurrency?” That usually means validating the session orchestration path around the avatar, not hand-rolling a browser benchmark for every test case.

For teams using a LiveKit-based voice agent, the Protoface plugin is the cleanest place to integrate the avatar into the existing agent pipeline. The plugin drops a synchronized talking face into the voice agent, so your load test can treat avatar rendering as part of the session lifecycle rather than as a separate app.

from livekit import agents

)

from livekit import agents

)

from livekit import agents

)

In practice, you want to test at the agent boundary: connect a virtual user, establish the voice session, attach the avatar, and then drive turns. That keeps the benchmark aligned with how the system is actually deployed. If you need the plugin details or examples, the integration repo and documentation are the right references: GitHub repo and docs.

Operationalizing the test run

Once the basic harness works, make it reproducible. Parameterize the test with environment variables so you can sweep concurrency and think time without editing code:

LOCUST_USERS=50 LOCUST_SPAWN=5 LOCUST_HOST=ws://localhost:8000 locust -f locustfile.py

LOCUST_USERS=50 LOCUST_SPAWN=5 LOCUST_HOST=ws://localhost:8000 locust -f locustfile.py

LOCUST_USERS=50 LOCUST_SPAWN=5 LOCUST_HOST=ws://localhost:8000 locust -f locustfile.py

Then run a small matrix:

• single user, baseline latency,

• 10-20 users, verify correctness and resource shape,

• ramp until p95 turn latency crosses your product threshold,

• hold at that level for 10-30 minutes to catch leaks and queue buildup.

Use the same test to compare deployment changes: more worker processes, a different event loop policy, a new STT model, or a different avatar quality tier. The point of the benchmark is not to chase the highest number; it is to find the point where the user experience stops being acceptable.

If you need to automate avatar/session setup outside the agent process, the REST API is a straightforward way to do it. A simple authenticated request looks like this:

curl -X POST https://api.protoface.com/... \
-d '{"...":"..."}'

curl -X POST https://api.protoface.com/... \
-d '{"...":"..."}'

curl -X POST https://api.protoface.com/... \
-d '{"...":"..."}'

The exact fields depend on the endpoint you are using, so keep this at the level of your documented contract. The useful pattern is the same: create the session before the load test, inject its identifiers into the harness, and validate that your system can sustain the required concurrent active sessions.

Conclusion

Load testing voice and video customer support agents is mostly about modeling the conversation lifecycle accurately: open long-lived connections, simulate realistic turn behavior, and measure latency at the points users actually perceive. FastAPI gives you a controlled target, WebSocket clients let you reproduce session behavior, and Locust makes it practical to ramp concurrency and observe where the system breaks.

If your stack includes realtime avatars, test them as part of the session path, not as an isolated video benchmark. Keep the harness simple, make the traffic representative, and focus on p95/p99 turn latency, connection stability, and teardown behavior.

For integration details, examples, and API contracts, start with the docs at docs.protoface.com. If you want working agent quickstarts, the repository linked from the project page is a good starting point.