Building a Realtime Voice + Video Receptionist for Your Website in Next.js
Build a realtime voice + video receptionist in Next.js with WebRTC, short-lived sessions, and synced avatar playback.
Introduction
If you want a receptionist on your website that can answer questions in real time, speak naturally, and show a synchronized talking face, you need to solve a small stack of problems at once: low-latency audio transport, live video rendering, turn-taking, and safe integration into a web app. In practice, this usually means building a voice agent first and then attaching a video avatar that stays in sync with the agent’s audio output.
This post walks through the core architecture for a realtime voice + video receptionist in Next.js. By the end, you should understand how to wire a browser client to a voice backend, where WebRTC fits, how to keep secrets out of the browser, and how to add a lip-synced avatar without turning your app into a media pipeline project.
Start with the transport, not the avatar
A receptionist is only useful if it responds quickly. That means your first design choice is the media transport, not the UI. For realtime conversational systems, the browser typically captures microphone audio, sends it to a backend over WebRTC or a similar low-latency streaming path, and receives synthesized audio back. If you want a video face, the avatar stream must be derived from the same turn-taking state as the audio; otherwise the mouth movements drift and the experience feels broken.
In a Next.js app, the browser component usually does three jobs:
request microphone permission and capture audio,
establish a session with your backend or provider,
render the remote video track or iframe embed that represents the avatar.
The backend should own anything sensitive: API keys, session creation, tenant routing, instructions, and rate limiting. Do not let the browser mint its own long-lived credentials. Even if you are only prototyping, treat the client as untrusted.
Build the Next.js shell around a realtime session
A clean implementation usually has one server endpoint that creates a short-lived session, and one client component that connects to it. In Next.js, that often means an API route or server action that returns the session metadata required by your realtime provider.
At a minimum, the server should:
authenticate the user if your receptionist is tenant-specific,
select the right voice, prompt, or business rules,
create or fetch a session,
return only the ephemeral data the browser needs.
The browser can then open the audio/video connection and render the remote media. If your agent stack already uses LiveKit or a similar media layer, this is the right place to keep the agent logic and avatar transport aligned.
On the client, you typically initialize the media session in a client component, then attach the avatar video element or iframe to the DOM. Keep the UI state minimal: connecting, connected, listening, speaking, disconnected, error. Anything more elaborate tends to leak transport details into the presentation layer.
A few implementation details matter more than people expect:
Audio-first UX: the user should be able to talk before the avatar fully renders. Don’t block audio on video startup.
Echo control: if the receptionist speaks back through the browser, make sure you are not re-capturing its output from the mic path.
Session lifecycle: handle reconnects explicitly. WebRTC sessions fail more often than HTTP requests.
Latency budget: once you exceed a few hundred milliseconds end-to-end, turn-taking starts to feel unnatural.
Where the avatar fits in the media pipeline
The avatar is not a separate “feature” bolted onto the UI. It is a media source that should track the agent’s speaking state. When the agent is generating audio, the avatar should render mouth motion from the same timing information. When the agent is listening, the avatar should idle or animate minimally. This synchronization is what prevents uncanny lag between speech and lip movement.
There are two common integration patterns:
Remote video track: the avatar is delivered as a live video stream and rendered like any other WebRTC participant.
Iframe embed: the avatar experience is hosted outside your app and embedded into your page, which keeps the browser integration simple and keeps API keys off the client entirely.
If you are already operating a voice agent backend, the first pattern gives you the most control. If you want a faster integration with minimal backend work, the second pattern is usually the shortest path to production.
Keeping secrets and tenancy boundaries out of the browser
For a receptionist, the security model matters. Website visitors are untrusted by default, and a public-facing widget should not expose long-lived credentials, internal prompts, or admin APIs. The usual pattern is to have your server issue short-lived session data or delegate the whole interaction to an isolated embed.
Two things to get right:
Authentication boundary: browser code should never contain your service API key.
Tenant boundary: if you support multiple customers or websites, make sure session creation is scoped per origin, per workspace, or per customer account.
Rate limits are not optional here. Realtime agents can be surprisingly expensive under abuse, especially if they involve high-quality audio synthesis and video generation. Put limits at the session layer, not only at the HTTP edge.
A practical Protoface integration path
If your goal is specifically a synchronized talking face for a voice agent, Protoface fits cleanly into the media layer rather than the UI layer. For a Next.js receptionist, the most straightforward option is often the customer-managed iframe embed: your page includes an iframe, while Protoface handles the realtime avatar session behind the scenes. That keeps the API key out of the browser and gives you a simpler deployment story for a public website.
For teams that already have their own voice backend, the REST API can be used to create and manage avatars and realtime sessions from your server. The auth model is standard bearer-token auth, so the browser never needs direct access to the key:
If your stack is Python-heavy, the SDK is the natural place to create sessions from backend code. And if your voice agent is built on LiveKit, the quickstart examples and the LiveKit plugin give you a direct path to attach a synchronized face to an existing agent without rewriting the agent logic. The same general rule applies across all of these surfaces: keep the agent, the media session, and the avatar in the same lifecycle so turn-taking stays consistent.
Operational concerns: quality, cost, and failure modes
Once you move past a demo, the non-obvious work is mostly operational. Realtime avatars introduce cost and failure modes that ordinary web widgets do not.
Quality tier matters because it changes your latency/cost trade-off. For a receptionist, that trade-off is usually acceptable as long as the first response feels immediate and the face stays synchronized. But you should still test with real network conditions, not local loopback. Mobile browsers, corporate proxies, and noisy microphones all expose different weaknesses.
Common failure modes to plan for:
ICE connection failures: WebRTC sessions may need retry logic or fallback handling.
Permission denial: users can decline mic access; the UI should degrade gracefully.
Speaking overlap: if the agent and user talk at once, your turn-taking policy should define who wins.
Session expiration: long conversations need reconnect or renewal handling.
In production, I would instrument at least four metrics: connect time, round-trip turn latency, audio interruption rate, and session drop rate. Those will tell you more about user experience than a raw “requests per minute” graph ever will.
Conclusion
Building a realtime receptionist in Next.js is mostly an exercise in clean boundaries: keep credentials on the server, keep transport logic separate from presentation, and make sure the avatar is driven by the same realtime session as the voice agent. If you do that, the browser code stays small and the user experience stays coherent.
For implementation details, session fields, and supported integration patterns, start with the docs. If you want a quicker path from prototype to working deployment, the quickstart repositories linked from the project readme are a good way to see the full flow end to end. The main thing is to treat the avatar as part of the media system, not as a decorative widget, and design the session lifecycle accordingly.