What Is a Realtime Avatar for Hospitality? Building Android Travel Agents with Kotlin
Explore realtime avatars for hospitality: Android/Kotlin architecture, low-latency voice pipelines, and Protoface integration.
Introduction
In hospitality, the UI is often the product: guests ask a question, need a recommendation, or want to change a booking, and the interaction has to feel immediate, clear, and low-friction. A realtime avatar is the visual layer for that interaction: a synchronized talking face that listens, responds, and renders an agent as something more natural than a static chatbot widget.
For developers, the interesting part is not the face itself. It is the pipeline behind it: speech input, turn detection, agent reasoning, text or speech generation, and a low-latency video stream that stays aligned with the voice. By the end of this post, you should be able to reason about that pipeline, understand where the hard edges are, and know how to embed an avatar into an Android travel agent without treating it like a gimmick.
What “realtime avatar” actually means
In practical terms, a realtime avatar is a media endpoint that behaves like a participant in a conversation. It is not a pre-rendered video clip. It is typically driven by one of three inputs:
Live audio from the user, used for speech recognition and turn-taking.
Agent output, usually text or audio, used to produce spoken responses.
Timing signals, so the avatar’s mouth shapes, head motion, and gaze stay aligned with the generated audio.
The important property is synchronization. If the voice says “your check-in is at 3 PM” while the face is still moving for the previous sentence, the illusion breaks immediately. That means the avatar layer has to accept streamed audio or tightly chunked speech and update the video frame cadence fast enough that the mouth looks causally related to the sound.
In a hospitality workflow, this matters more than it first appears. Guests tolerate delays poorly when they are asking about arrival time, late checkout, shuttle availability, or property policies. A realtime avatar can keep the interaction feeling responsive even when the backend agent is doing nontrivial work such as searching reservations or calling a property system.
Android architecture: keep media local, keep logic modular
If you are building an Android travel agent, the cleanest architecture is usually:
The app captures microphone input and sends it to your voice agent stack.
The agent performs ASR, reasoning, and response generation.
The avatar stream is attached as a visual participant, or rendered as an iframe if you are using a web surface inside the app.
The Android client handles session lifecycle, token acquisition, network state, and UI orchestration.
Do not bury business logic in the avatar layer. The avatar is presentation and presence. Booking lookups, hotel policy retrieval, loyalty status, and escalation logic should stay in your agent service or backend. That separation matters for testability, compliance, and retries.
Low-latency constraints you actually need to design for
A travel agent feels good when it responds in under a couple of seconds for routine questions and degrades gracefully for slower operations. That means you need to think about three separate latencies:
Input latency: how quickly audio is turned into intent or text.
Agent latency: how long reasoning and tool calls take.
Media latency: how long it takes to convert the agent’s response into synced voice and video.
For hospitality, the third item is often underestimated. Even if your LLM is quick, the perceived experience can still feel sluggish if the avatar starts “speaking” after the text is ready but before the audio is actually playable. In practice, you want streaming responses and a pipeline that begins animating as soon as the first chunks of synthesized speech are available.
On Android, that also means being careful with lifecycle events. If the app backgrounds, rotates, or loses connectivity, the session should either reconnect cleanly or fail in a way that does not leave the user in a broken conversational state. Treat avatar sessions like any other realtime media session: explicit connect, heartbeat, resume or teardown, and clear error handling.
Building the agent around hotel-specific turns
Travel and hospitality agents have a few interaction patterns that are worth designing for up front:
Short factual queries: “What time is breakfast?”
Stateful tasks: “Change my check-out from Friday to Saturday.”
Ambiguous requests: “Can I get a quieter room?”
Escalations: “I need someone at the front desk.”
These are not all the same problem. Short factual queries can be answered quickly from a knowledge base or property policy store. Stateful tasks need confirmation and often tool calls. Ambiguous requests benefit from clarification before the agent acts. Escalations should preserve context and hand off cleanly.
A useful pattern is to make the agent produce structured intents and then generate natural language only after the action is confirmed. That reduces the risk of the avatar confidently saying something that the backend cannot support. For example, if the user asks for a late checkout, the agent should check eligibility before announcing success.
Keeping the avatar honest: synthesis, turn-taking, and fallback behavior
Realtime avatars are most convincing when they respect turn-taking. The agent should not interrupt itself, and the avatar should not continue animating after the response is complete. If your speech stack supports endpointing, tune it carefully; if it does not, add explicit markers in your agent loop so the avatar knows when to start, pause, and stop.
You also need a fallback path. Hospitality users will forgive a slower textual answer more readily than a broken media stream. If video delivery fails, the agent should still provide voice or text. If voice fails, the transcript should remain visible. The avatar is a layer on top of the interaction, not the interaction itself.
This is especially important on Android because network variability is normal. A lobby Wi-Fi network may be unstable; a guest may move between cellular and Wi-Fi; a device may enter power-save modes. Design your session state machine so that the agent can continue operating even if the visual layer temporarily disconnects.
Protoface in the stack
This is where Protoface fits cleanly: it gives you a developer-facing realtime avatar layer that you can attach to a voice agent without building the video presence system yourself. For an Android travel agent, the most natural integration path is usually to keep your agent logic where it already lives and attach an avatar session to the conversation.
If you are using a Python-based agent backend, the SDK is the fastest way to create and manage sessions programmatically. The exact fields are documented, but the shape is familiar: authenticate with an API key, create an avatar/session, then connect your agent media loop to that session.
For teams already built around LiveKit Agents, the plugin path is even simpler. The livekit-plugins-protoface package drops a synchronized face into the agent stream so you can keep the rest of your voice stack intact. That is usually the least disruptive option if you already have ASR, TTS, tool calling, and turn management wired up.
If you need to inspect or automate from outside your application, the REST API is straightforward: create and manage avatars and realtime sessions with bearer authentication. A minimal request looks like this:
For implementation details, quotas, and session fields, use the documentation. If you want a concrete starting point, the GitHub organization has quickstarts that show how these pieces fit into an agent stack.
Android-specific implementation notes
On Android, keep the client thin. The app should handle login, device permissions, session start/stop, and UI state. The heavy lifting belongs in your backend or agent service. That avoids shipping API keys into the app, which is especially important if you are controlling sessions directly.
A few implementation details are worth calling out:
Permission flow: request microphone access only when the user enters the conversation surface.
Session isolation: one user session should map to one agent context; do not reuse sessions across users.
Retry strategy: reconnect media streams idempotently when possible, but do not silently duplicate agent turns.
UI feedback: show whether the agent is listening, processing, or speaking, because motion alone is not enough.
If your Android app is primarily a wrapper around a web experience, an iframe-based avatar can be a pragmatic route. The key advantage is that the browser surface owns the media behavior and can enforce origin restrictions and other guardrails without exposing a backend secret in the client. That is useful when you want to move quickly or keep the mobile app lean.
Operational concerns: security, rate limits, and observability
Realtime avatars introduce a few operational concerns that are easy to miss until production:
Auth: keep server credentials server-side. Use bearer authentication for API calls from trusted services only.
Rate control: prevent abusive sessions, especially in public guest-facing environments.
Observability: log session starts, disconnects, turn durations, and tool failures so you can debug bad conversational turns.
Cost discipline: choose quality tier intentionally; better media quality is useful, but it should match the deployment surface and expected traffic.
For hospitality specifically, observability matters because the failures are often semantic, not just technical. A guest may say “I think my booking is under Patel,” and the agent might misread the name, search the wrong reservation, or answer with an overconfident refusal. Good logs let you distinguish recognition problems from backend lookup problems.
Conclusion
A realtime avatar for hospitality is not just a video face. It is a synchronized media surface attached to an agent that has to manage speech, timing, state, and failure modes well enough to feel reliable in a guest-facing setting. On Android, the cleanest design is usually to keep your mobile client focused on permissions and session orchestration, while your backend owns the conversational logic and tool calls.
If you want to try this for a travel or hotel agent, start with a minimal voice workflow, add the avatar as a separate concern, and verify that turn-taking and fallback behavior are solid before tuning visual polish. The docs at docs.protoface.com are the right place to check the exact API shapes, and the quickstarts linked from the GitHub org are useful when you want a working reference before you wire it into your own stack.