Rust + LiveKit: Building a Realtime Avatar for Real Estate Lead Qualification
Build a LiveKit realtime avatar for real-estate lead qualification with Rust, sync speech/video, and secure backend session management.
Introduction
Real-estate lead qualification is a good fit for a voice agent, but a plain audio bot is often not enough. Buyers and renters expect a fast conversational experience, yet the interaction still benefits from a visible face: it increases engagement, makes turn-taking clearer, and gives the agent a more polished presence on your site or in your call flow.
The problem is that “add a talking face” is not a UI exercise; it is a realtime systems problem. You need low-latency speech, synchronized lip motion, predictable session lifecycle, and a way to keep the avatar aligned with the agent’s response stream without leaking credentials into the browser.
By the end of this post, you should understand how to wire a realtime avatar into a LiveKit voice agent for a lead-qualification workflow, how the streaming pieces fit together, and where the operational edges are: latency, session boundaries, and authentication.
What “realtime avatar” means in practice
For developers, a realtime avatar is not a pre-rendered video clip. It is a video surface that is driven by the same token stream or speech stream your voice agent emits. The avatar should begin speaking as soon as the agent has something to say, and its mouth motion should stay in sync with the speech audio, not drift behind it.
That implies a few constraints:
Low end-to-end latency. If you add too much delay between model output and avatar rendering, the conversation feels laggy.
Shared session state. The avatar session should be tied to the same conversation context as the agent.
Deterministic handoff. When the agent starts speaking, the avatar should switch from idle to talking with no extra application logic scattered across your codebase.
WebRTC-friendly delivery. Video and audio need a transport that tolerates realtime media and NAT traversal.
In a lead-qualification flow, that means your app can ask a few structured questions — budget, location, timeline, financing, desired property type — while the avatar provides a consistent front-end to the interaction. The implementation should feel more like attaching a media track to an existing agent than like building a separate video product.
How the LiveKit agent architecture fits
LiveKit is a sensible place to build this because it already gives you a realtime media graph: participant state, audio transport, and an agent runtime that can publish and subscribe to tracks. The avatar becomes another track in that graph, driven by the agent’s speech.
The basic shape is:
A user connects to a LiveKit room and speaks.
Your agent receives audio, transcribes it, decides what to say next, and generates text and/or speech.
The avatar layer consumes the agent’s speech output and renders a synchronized talking face.
The video track is published back into the room, so the user sees and hears a single cohesive assistant.
For real estate qualification, this is useful because the conversation usually needs just enough structure to collect contact info and intent, not a full app shell. The avatar gives the agent a stable visual identity while the agent runtime handles branching questions, memory, and routing to a human when needed.
Building the flow with a LiveKit plugin
If you are already using LiveKit Agents, the most direct integration is the Protoface plugin. The idea is simple: drop the avatar service into the agent pipeline so the voice agent gets a synchronized face without you manually stitching together a video renderer.
A minimal setup looks roughly like this:
The important architectural point is not the exact constructor shape — check the docs for the current API — but the placement in the stack. The avatar should sit close to the agent’s speech generation, not as a separate post-processing step in your frontend. That keeps the visual output aligned with whatever the agent is actually saying.
A few practical notes:
Keep the prompt narrow. Lead qualification works better when the agent is constrained to a fixed questionnaire and a clear handoff policy.
Separate business logic from presentation. The avatar should not know about CRM fields or qualification scoring. It should just reflect the agent’s spoken output.
Plan for interruption. If the user interrupts the agent mid-sentence, the audio stream and avatar state must stop cleanly before the next turn starts.
Session management and authentication from your backend
In production, you usually want your backend to create or manage avatar sessions rather than exposing long-lived credentials to the client. That keeps authorization server-side and lets you attach session metadata, usage limits, or business identifiers before the interaction starts.
Protoface exposes a REST API for avatar and session management, authenticated with API keys. A simple server-side call pattern looks like this:
The exact request schema is documented in the API reference, but the operational idea is what matters: create the session on your server, return only the minimal session artifact your client needs, and keep the API key out of the browser. That matters even more for a lead-qualification use case, because the same application often touches PII: names, phone numbers, budgets, and preferred neighborhoods.
If you prefer to manage this from Python, the SDK gives you the same control from application code:
That is usually enough for a backend service that provisions sessions on demand, logs them, and correlates them with your CRM or analytics pipeline.
Why this matters for lead qualification specifically
Lead qualification has a few properties that make realtime avatars useful and also easy to get wrong.
First, the interaction is short and repetitive. Users are answering a predictable set of questions, so the product wins come from reducing drop-off, not from novel conversational depth. A visible avatar helps the flow feel intentional without adding cognitive load.
Second, the business outcome is usually “capture enough context to route the lead.” That means you need reliability more than personality. It is better to have a slightly conservative avatar pipeline that stays in sync than a flashy one that occasionally desynchronizes or fails to connect.
Third, the handoff to a human matters. The avatar should not obscure whether the user is speaking to automation. If the agent detects a high-intent lead, a financing question, or an edge case, you should be able to stop the agent cleanly and transfer the room, transcript, or lead summary to a human rep.
In other words, the avatar is an interface layer over a structured qualification workflow. The goal is not “make it look animated”; the goal is “make the conversation easier to start and easier to trust.”
Where Protoface fits
This is exactly the kind of integration Protoface is built for: a developer-facing avatar layer that can be attached to a realtime voice agent without reimplementing media synchronization yourself. For LiveKit users, the plugin approach is the shortest path; the quickstart examples are useful if you want to see the surrounding agent setup, and the docs cover the current session and API shapes.
The key design choice is that the avatar is treated as part of the agent runtime, not as a front-end gimmick. That keeps the browser thin, lets your backend own authentication, and makes it easier to reason about lifecycle: create session, connect agent, stream speech, publish video, then tear everything down when the lead qualifies or the user leaves.
Common implementation gotchas
A few issues come up repeatedly in production:
Overlong turns. If the agent monologues too much, users stop responding. Keep qualification responses short and structured.
Prompt drift. If the model starts improvising, the avatar can look polished while the workflow becomes inconsistent. Use fixed steps and explicit stop conditions.
Session leaks. Make sure abandoned browser sessions close cleanly so you do not keep paying for stale realtime rooms or avatar sessions.
Latency stacking. STT, LLM, TTS, and avatar rendering each add delay. Measure the full chain, not just model inference time.
For debugging, it helps to log the turn timeline: user speech start, transcript ready, model response start, avatar speech start, and session end. If you cannot explain the time between those events, users will feel it before you can measure it.
Conclusion
A realtime avatar is most valuable when it is treated as part of the agent pipeline, not as a separate media feature. For real-estate lead qualification, that means a short, structured conversation with a visible face, minimal browser logic, and backend-owned session control.
If you are building this on LiveKit, start with the plugin path and keep the rest of your system boring: explicit session creation, narrow prompts, and clear handoff behavior. When you are ready to wire it up, the implementation details and current API surface are in the documentation, and the quickstart repositories are a good way to validate the end-to-end flow before you integrate it into production.