Introduction

Healthcare intake is a good stress test for any voice bot. Users speak in fragments, change answers midstream, pause to think, and expect the system to keep context across a long conversation. A plain audio-only agent can handle the conversation, but it often feels brittle because the interaction lacks visual grounding. A realtime avatar changes that dynamic: you keep the same conversational loop, but the user also gets a synchronized face that reinforces turn-taking, makes interruptions less awkward, and generally reads as a more deliberate product.

This post shows how to migrate an existing healthcare intake bot from text-and-voice logic to a realtime avatar experience without rewriting the core agent. The goal is practical: keep your speech pipeline, preserve your intake state machine, and add a lip-synced avatar layer that can be deployed in a voice app or embedded on the web.

Start with the agent architecture you already have

Most intake bots already follow a structure like this:

• ASR turns user speech into text.

• The LLM extracts intent and updates the intake state.

• TTS produces the response audio.

• The application manages turn-taking, retries, and escalation.

The avatar should not replace any of that. It sits downstream of the agent’s response stream and renders the same spoken output as a live face. That distinction matters because the avatar should not be making clinical decisions, parsing structured fields independently, or inventing dialogue. It should mirror the already-decided agent output with minimal additional latency.

For healthcare workflows, this separation also keeps your compliance and auditing story cleaner. The conversational logic remains in your agent. The avatar is presentation.

What changes in a realtime avatar migration

The key technical shift is that your response path is no longer just “text to audio.” You now care about synchronized multimodal output:

1. The agent emits a response.

2. The speech layer starts synthesizing audio.

3. The avatar receives the same turn and animates the face in sync with the audio stream.

4. The client renders the video over a realtime transport, usually WebRTC.

In practice, this means you want low end-to-end latency and a stable session model. If the face lags the audio by too much, users notice immediately. If the connection drops or the session is recreated too often, the illusion of a live assistant breaks. So when you migrate, pay attention to these areas first:

• Turn boundaries: the avatar should animate only on finalized agent turns, not while the model is still deliberating.

• Audio-video sync: the lip motion should follow the generated speech, not a separate timer.

• Session state: reuse the same conversation context across reconnects if your use case allows it.

• Interruptions: decide what happens when the patient starts speaking mid-response. Most intake bots should stop talking promptly and yield the floor.

Step 1: Keep the intake logic isolated from presentation

Before adding an avatar, make sure your agent exposes a clean response interface. Ideally you already have something like:

def handle_user_utterance(text: str, state: dict) -> dict:
...

def handle_user_utterance(text: str, state: dict) -> dict:
...

def handle_user_utterance(text: str, state: dict) -> dict:
...

That interface should remain the source of truth. The avatar layer should consume only the final response text or the finalized speech turn, plus any metadata needed for timing or persona configuration. Avoid coupling avatar state to your schema extraction code. If the face becomes responsible for logic, debugging gets painful fast.

For healthcare intake specifically, I recommend treating the conversational turn as immutable once it is spoken. If the agent needs to correct itself, emit a new turn rather than mutating the old one. That makes transcript storage, QA review, and safety checks easier.

Step 2: Add the avatar as a synchronized output channel

If you are running a voice agent in LiveKit, the cleanest migration path is to drop in a plugin that adds the avatar as another realtime output surface. The agent still handles ASR, LLM, and TTS; the plugin mirrors the spoken turn into a synchronized talking video face.

Example sketch:

# illustrative only; see the package docs for exact config fields

session.run()

# illustrative only; see the package docs for exact config fields

session.run()

# illustrative only; see the package docs for exact config fields

session.run()

The details vary by stack, but the integration pattern is the same: attach the avatar where the final spoken output is produced. If your voice pipeline already includes barge-in handling, preserve that behavior. The avatar should stop and restart with the audio turn so the facial motion matches the actual spoken content.

For LiveKit users, the plugin repository is the most relevant place to start: GitHub examples and integration notes. If you are using Pipecat instead of a direct LiveKit agent, the integration follows the same principle but is wired into the Pipecat service graph.

Step 3: Manage sessions explicitly

In a healthcare setting, you should think about avatar sessions the same way you think about agent sessions: they are stateful resources with lifecycle, limits, and cleanup. A session needs to represent one active interaction, not a long-lived browser tab that can accidentally duplicate connections.

At minimum, define how you will:

• create a session when the intake starts,

• resume or reconnect if the client refreshes,

• end the session when the intake is complete or abandoned, and

• store enough metadata to map sessions back to the patient flow.

Here is a lightweight example using the REST API shape you would expect for session creation. The exact fields are documented, so treat this as a pattern rather than a copy-paste contract:

curl -X POST https://api.protoface.com/sessions \
}'

curl -X POST https://api.protoface.com/sessions \
}'

curl -X POST https://api.protoface.com/sessions \
}'

That session can then be used by your voice agent or web client to join the realtime avatar stream. The important architectural point is that the browser or frontend should never need your API key. If you are embedding directly in a site, use an approach that keeps server credentials off the client entirely.

Step 4: Keep the web deployment secure and boring

For web-based intake or pre-visit screening, an iframe embed is usually the fastest path. The practical advantage is not just convenience; it is security boundary control. You can allowlist the parent origin, rate limit usage per IP or duration, and pass per-embed voice or instructions without exposing any backend credentials in the browser.

That matters in healthcare because frontends tend to get copied across environments. A simple embed pattern prevents accidental leakage of API keys and reduces the amount of custom client code you need to audit.

Operationally, the embed model also makes rollout safer:

• you can ship a single hosted avatar experience,

• embed it into a patient intake page,

• swap the persona or instructions server-side, and

• roll back without pushing a frontend release.

If you are building a custom UI, make sure the avatar container handles network reconnects gracefully. WebRTC sessions can fail for ordinary reasons: browser sleep, mobile tab switching, restrictive proxies, or a patient moving from Wi-Fi to cellular. Your app should recover the session or degrade cleanly back to audio-only rather than leaving a frozen face on screen.

Where Protoface fits in this migration

Protoface gives you the avatar layer without forcing you to replace the rest of the intake stack. For a LiveKit-based bot, the plugin path is the shortest route: keep your agent, attach the avatar, and let the existing speech pipeline drive the synchronized video face. If you want to provision avatars or manage sessions directly, the REST API and Python SDK are the right surfaces. If you are embedding the intake experience in a website, the iframe model avoids exposing credentials to the browser and is easier to govern in production.

For implementation details, the public docs are the source of truth: docs.protoface.com. If you want to see the plugin and quickstart patterns in code, the repo organization under GitHub is the most useful reference point.

Gotchas specific to healthcare intake

There are a few failure modes worth calling out explicitly:

• Latency budgeting: the avatar should not add enough delay to make the user repeat themselves. Track the full chain: ASR, LLM, TTS, avatar rendering.

• Interrupt handling: barge-in should stop both audio and animation quickly.

• Transcript accuracy: keep a canonical transcript independent of the rendered face so QA and compliance review have one source of truth.

• Instruction drift: the avatar should never improvise around clinical questions. The persona layer should be narrow and tightly controlled.

• Fallbacks: if the video path fails, degrade to voice-only rather than ending the intake.

One useful pattern is to gate the avatar by workflow stage. For example, you may want the face during greeting, consent, and routine intake questions, but switch to voice-only during sensitive escalation or when the patient needs to enter protected details on their own device. That keeps the interaction appropriate without sacrificing the benefits of a visual agent where it helps most.

Conclusion

Migrating a healthcare intake bot to a realtime avatar is mostly an integration exercise, not a rewrite. Keep the agent logic where it belongs, attach the avatar as a synchronized presentation layer, manage sessions deliberately, and treat latency and barge-in as first-class concerns. If you do that, you get a more usable intake experience without compromising the conversational core.

For the implementation details, start with the docs and the relevant integration repo, then prototype the avatar on top of your existing agent before changing the rest of the stack. The quickest path is usually: wire the voice agent, attach the avatar, measure latency, and only then tune persona and deployment options.