Introduction

Adding a talking virtual receptionist to a Next.js app sounds simple until you try to make the pieces work together: a browser UI, a realtime voice pipeline, low-latency video generation, authentication, and session lifecycle management. The hard part is not rendering a face; it is keeping audio, video, and agent state synchronized well enough that the interaction feels conversational instead of stitched together.

This post shows a practical path for integrating a virtual receptionist into a Next.js front end with a LiveKit-based voice agent and a FastAPI backend. By the end, you should understand how to:

• create or select an avatar server-side,

• start a realtime session from your backend,

• connect the session to a LiveKit agent,

• stream the avatar into the browser with low latency, and

• avoid the common security and sequencing mistakes that make these systems fragile.

Architecture: keep the browser thin

The first design choice is important: do not let the browser talk directly to the avatar API with long-lived credentials. The browser should only receive short-lived, scoped connection data. Your FastAPI service should own API keys, session creation, and any business logic for when a receptionist should be spawned, transferred, or torn down.

A clean architecture looks like this:

• Next.js: renders the receptionist UI, requests a session token from your backend, and connects to LiveKit.

• FastAPI: authenticates your user, creates the avatar/session on your behalf, and returns only the data the browser needs.

• LiveKit agent: handles the voice conversation and pushes audio into the avatar pipeline.

• Avatar service: generates synchronized talking video from the agent’s audio stream.

In practice, this separation keeps secrets off the client, makes auditing easier, and gives you one place to enforce rate limits, tenant rules, and session cleanup.

Backend session creation with FastAPI

For a receptionist flow, the browser usually asks your backend for a new session when the page loads or when the user clicks “Start call.” Your backend then talks to the avatar API using its own API key. The exact request shape depends on your workflow, so treat the payload below as illustrative rather than copy-paste complete.

from fastapi import FastAPI, Depends

return resp.json()

from fastapi import FastAPI, Depends

return resp.json()

from fastapi import FastAPI, Depends

return resp.json()

A few implementation notes:

• Keep the API key in server-side environment variables only.

• Use your own user/session identity to decide which avatar or instructions to apply.

• Return only the browser-facing fields: for example, a session id, ephemeral token, and any LiveKit connection details your frontend needs.

If you prefer a direct SDK flow, the Python SDK can do the same orchestration from your backend without manually building requests. The details vary by version, but the pattern is the same: create the avatar/session server-side, then hand the browser a minimal connection object. See the SDK repository and the docs for the exact method names and fields: https://github.com/protoface-ai/protoface-sdk-python and https://docs.protoface.com.

Connecting the LiveKit voice agent to a talking face

The real lift happens in the agent process. A LiveKit voice agent already owns the conversation loop: it receives microphone audio, performs speech-to-text, decides on a response, generates speech, and streams audio back. The avatar plugin sits alongside that pipeline and consumes the generated speech so the face can lip-sync to the same utterance the user hears.

That means the avatar is not “doing AI” separately. It is visually tracking the output of the voice agent. This matters because you want the mouth movement to follow the same utterance boundaries and timing as the audio, not a second independent model.

from livekit.agents import WorkerOptions, cli

cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

from livekit.agents import WorkerOptions, cli

cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

from livekit.agents import WorkerOptions, cli

cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

The important operational details are:

• Single source of truth for speech timing: the same agent output that drives audio should drive the face.

• Backpressure matters: if the agent is producing audio faster than the avatar pipeline can render, you need buffering or throttling, not a second parallel response path.

• Turn-taking must be explicit: for a receptionist, interruptibility, barge-in, and end-of-utterance handling are more important than flashy animation.

For developers already using Pipecat, the integration pattern is similar: the avatar becomes a video service attached to the voice pipeline. The plugin and service docs are useful references if your stack sits there instead of directly on LiveKit: https://pypi.org/project/pipecat-protoface/ and https://docs.pipecat.ai/api-reference/server/services/video/protoface.

Next.js client wiring: keep it minimal

On the frontend, avoid overengineering. Your Next.js page should:

1. call your FastAPI endpoint to create or fetch a receptionist session,

2. connect to the LiveKit room using the returned ephemeral credentials,

3. render the video track for the avatar, and

4. provide a simple audio/video control surface for the user.

Do not make the browser responsible for avatar lifecycle. It should not know how to create sessions, select avatar presets, or manage API keys. If the page reloads, the backend can decide whether to resume the session or create a fresh one.

A useful mental model is that the browser is a terminal for conversation, not the conversation engine itself.

Operational gotchas that matter in production

Once you have a demo working, the real work is usually around reliability and safety.

• Latency budget: the user experiences the sum of STT, LLM, TTS, avatar rendering, and transport. Each hop should be measured, not guessed.

• Cold starts: if your receptionist can be summoned on demand, pre-warming the agent worker or keeping a small pool alive often helps more than micro-optimizing the frontend.

• Session cleanup: reclaim rooms, avatar sessions, and any associated ephemeral resources when the call ends or the browser disconnects.

• Security: never expose your API key in the browser; use backend-issued ephemeral credentials or embeds designed for client-side use only when that is the intended model.

• Prompt and voice consistency: a receptionist should sound deterministic enough to feel dependable. Keep instructions short and business-specific.

If you need a browser-embedded avatar without backend code, customer-managed iframe embeds are a valid alternative. They are specifically designed so you do not expose API keys in the browser, and they support parent-origin allowlists plus per-embed voice, instructions, and rate limits. That is a different trade-off than the LiveKit-agent path, but it is useful when the avatar is the product rather than a component.

Where Protoface fits

For this particular architecture, the most relevant surface is the LiveKit plugin plus the API behind it. The plugin gives your voice agent a synchronized talking face without forcing you to build a separate video generation pipeline, while the REST API and backend SDK let you create and manage sessions from FastAPI with your own auth and tenancy model. The practical benefit is that your agent remains a normal voice agent; the avatar is an attached realtime surface, not a second app to coordinate.

If you are evaluating implementation details, start with the docs and one of the quickstarts in the GitHub repo. The docs are the source of truth for payloads, auth, and session fields, and the quickstarts are the fastest way to see the control flow end to end.

Conclusion

The reliable way to add a talking virtual receptionist to a Next.js app is to keep the browser thin, let FastAPI own session creation and secrets, and attach the avatar to the voice agent so audio and lip-sync stay aligned. That approach scales better than trying to orchestrate everything client-side, and it maps cleanly onto a real production deployment.

Start with the docs at https://docs.protoface.com, then wire up one minimal backend endpoint and one LiveKit worker. Once that works, harden the session lifecycle, measure latency, and only then add richer UI behavior like transfers, hold states, or receptionist personas.