Introduction

If you are adding account support or payment flows to a SvelteKit app, the hard part is usually not the UI. It is the handoff between identity, support context, and a realtime conversation that feels responsive enough to trust. A fintech assistant has to do a few things well at once: listen, speak, show a face, keep latency low, and stay within strict security boundaries when money is involved.

This post walks through the architecture I would use for a voice+video assistant in SvelteKit, with a focus on account support and payment guidance. By the end, you should understand how to wire the frontend, where the realtime session lives, what belongs on the server, and how to avoid leaking API keys or overcomplicating the browser side.

Start with the right shape of the system

For fintech, the frontend should not directly orchestrate everything. SvelteKit can render the page, collect user intent, and start a session, but the actual agent flow should run in a trusted backend or a managed realtime layer. That keeps secrets off the client and makes it easier to enforce session policy.

At a minimum, you want four pieces:

• SvelteKit UI for the chat/voice entry point, account context, and consent prompts.

• Session backend to mint short-lived session credentials, select the right avatar, and attach metadata like user ID or case ID.

• Voice agent runtime to handle speech-to-text, LLM reasoning, and text-to-speech.

• Video avatar layer so the agent has a synchronized talking face instead of a disembodied voice.

In practice, you should treat the avatar as a presentation layer for the agent, not the agent itself. The avatar should follow the audio stream and mouth timing generated by the voice agent, with any expression or idle behavior driven by session state. That distinction matters because in support and payments flows, the “what” of the conversation is the agent; the “how it appears” is just one output channel.

SvelteKit: keep the browser thin

Your SvelteKit app should do three things well: authenticate the user, request a short-lived realtime session, and attach that session to a component that can render the video face and audio controls. Avoid shipping long-lived secrets to the browser. If you need to call your own backend or the Protoface REST API, do that from a server route, not from the client.

A common pattern is:

1. User opens the support page and signs in.

2. SvelteKit loads account context server-side.

3. The client asks your server for a session token or embed URL.

4. The browser joins the voice/video session with that ephemeral credential.

That gives you a place to enforce business rules. For example, you can block payment-related intents until the user reauthenticates, or limit the assistant to read-only account support unless the session has extra approval.

// +page.server.ts
}

// +page.server.ts
}

// +page.server.ts
}

Then, from a server endpoint, create the realtime session using your backend credentials. The exact request fields depend on the avatar/session model you choose, but the important point is that API keys stay on the server:

curl -X POST https://api.protoface.com/v1/sessions \
}'

curl -X POST https://api.protoface.com/v1/sessions \
}'

curl -X POST https://api.protoface.com/v1/sessions \
}'

In a SvelteKit app, you would typically proxy that through an endpoint and return only the minimal session payload the browser needs. If you are using an iframe embed, this step can disappear entirely on the client side because the iframe itself handles the interactive session without exposing a backend key in the browser.

Realtime behavior: what actually needs to be synchronized

A voice+video assistant is not just “audio plus a face.” The system has to keep several streams aligned:

• Audio output from text-to-speech.

• Lip sync / viseme timing so the mouth matches phonemes closely enough to feel natural.

• Turn state so the avatar knows when the user is speaking versus when the agent is speaking.

• Conversation metadata such as intent, risk level, or escalation state.

For finance, turn state matters more than it does in a generic chatbot. If the user is reading a card number, interrupting them with a premature response is bad UX and can create compliance problems. The assistant should listen fully, confirm sensitive actions explicitly, and avoid discussing or displaying unnecessary payment data.

From an implementation standpoint, the agent loop usually looks like this:

mic audio -> STT -> LLM/agent -> policy checks -> TTS -> avatar video sync

mic audio -> STT -> LLM/agent -> policy checks -> TTS -> avatar video sync

mic audio -> STT -> LLM/agent -> policy checks -> TTS -> avatar video sync

Two practical points:

• Use server-side policy checks before any payment-related action. The model should not be the final authority on whether a transfer, card update, or refund is allowed.

• Keep latency budgeted end to end. Even if each hop is “fast,” a support assistant that takes too long to answer feels unreliable. In voice UIs, small delays are much more noticeable than in text chat.

Payments and account support: design for narrow authority

For fintech, the assistant should be narrowly scoped. I would separate use cases into at least three categories:

1. Read-only account support: balances, transaction status, document help, FAQ, troubleshooting.

2. Guided payment flows: explain steps, confirm intent, collect user consent, and hand off to a secure payment component.

3. High-risk actions: card replacement, refunds, disputes, or wire changes, which usually need stronger verification or a human takeover.

Do not let the avatar become a general-purpose financial operator. Instead, let it act as a conversation layer that can explain, route, and confirm. When the user crosses into a sensitive action, switch to a controlled flow: reauthenticate, show a dedicated form, or escalate to an agent.

One useful implementation detail is to carry context in session metadata rather than in the prompt alone. That lets your backend make deterministic decisions about what the assistant is allowed to do. For example, a session started from a “billing question” page should not silently gain permission to initiate a payment review flow unless the server explicitly upgrades it.

Where Protoface fits

This is the part where Protoface is useful: it gives you a dedicated realtime avatar layer without asking you to build the video face pipeline yourself. For a SvelteKit fintech assistant, the most relevant surface is the customer-managed iframe embed or a backend-created session. The iframe route is simplest if you want a no-browser-secret integration; the REST API is the right fit if your app already has a server-side orchestration layer and you want to create sessions programmatically.

Because the iframe is customer-managed, you can put it behind an origin allowlist and configure per-embed voice, instructions, and rate limits. That matters for fintech, where you often want tight control over where the assistant can run and how long a session can stay open. If you prefer to create sessions yourself, the API and Python SDK are there for programmatic control, and the docs cover the exact payloads and supported fields.

If you are looking for the API reference and integration specifics, start with the docs. If you want to inspect quickstart patterns and sample wiring, the GitHub org has the relevant repos.

Embedding the assistant in SvelteKit

There are two implementation patterns that tend to work well in SvelteKit.

Pattern 1: server-minted session URL

Your SvelteKit backend creates a session, stores the mapping to the signed-in user, and returns a short-lived URL or token. The client renders a lightweight component that consumes it. This is the better choice when you need to attach internal account context, authorization state, or audit metadata.

Pattern 2: iframe embed

Your page renders an iframe that hosts the interactive avatar experience. The parent app can pass limited, non-sensitive configuration, but the browser never sees your backend API key. This is useful when the assistant is mostly a frontend feature and you want to minimize integration surface.

In both cases, keep the interface clean. The user should be able to start voice, see the avatar, and understand when the assistant is listening or speaking. For payments, also show a persistent visual indicator when the assistant is in a restricted mode, such as “support only” or “secure handoff required.” That makes the policy explicit instead of invisible.

<!-- Example shell in SvelteKit -->
</div>

<!-- Example shell in SvelteKit -->
</div>

<!-- Example shell in SvelteKit -->
</div>

If you are not using the iframe, your browser component should still be thin. Render the visual surface, connect audio devices, and react to session events. Do not embed business logic or secrets in client code.

Operational concerns you should plan for

Fintech support assistants need the same operational discipline as any other customer-facing realtime system, plus stricter boundaries around data and permissions.

• Auditability: log session starts, intent changes, and any handoff to a human or secure form.

• Rate limits: protect against abuse and replay, especially on publicly reachable embeds.

• Time limits: end sessions cleanly when the user goes idle or completes the task.

• Fallbacks: if the avatar or media session fails, degrade to text chat or a support ticket, not a blank panel.

You should also plan for latency and partial failure. WebRTC media may connect while your app backend is still resolving account state, or vice versa. Make session state explicit in the UI so the user knows whether they are waiting for the assistant, waiting for account lookup, or already connected.

Conclusion

The practical way to add a fintech voice+video assistant in SvelteKit is to keep the browser thin, keep sensitive policy on the server, and treat the avatar as a realtime presentation layer for the agent. That gives you room to handle account support and payment guidance without exposing secrets or letting the model make unchecked financial decisions.

If you want to implement this next, start with a narrow support flow, then add voice, then add the video face once the session and policy boundaries are solid. The docs at docs.protoface.com cover the available surfaces and the exact payloads. From there, you can decide whether your integration is best served by a backend-created session, a plugin-based voice agent, or a customer-managed iframe embed.