Introduction

If you are building a realtime AI shopping assistant, you are usually stitching together three very different trust boundaries: a public browser client, a backend that can call LLMs and commerce APIs, and a realtime media layer that streams audio/video over WebRTC. Security breaks when those boundaries blur. The classic failure mode is exposing an API key in the browser because “it’s just a demo,” or letting a frontend mint privileged sessions because “the assistant needs to start fast.”

The goal here is to build a model that is safe by default:

• API keys stay server-side and are used only from trusted backend code.

• JWTs identify end users or sessions, but do not grant broad platform privileges.

• Role-based access control (RBAC) limits what each caller can do: customer, support agent, admin, service account.

• Realtime avatar sessions can be created, attached to voice agents, and embedded without turning your browser into a secret vault.

By the end, you should be able to design a shopping assistant that can talk, show a face, and access inventory or order data without making your auth model brittle.

Start with the trust boundaries

For a shopping assistant, think in terms of three layers:

1. Browser: untrusted. Assume JavaScript can be inspected and requests can be replayed.

2. Application backend: trusted for business logic, token minting, and server-to-server calls.

3. Realtime/avatar service: trusted infrastructure that should only accept authenticated, narrowly scoped requests.

That immediately gives you the rule that matters most: never put a long-lived platform API key in the browser. API keys are for backend-to-platform authentication only. If you need the browser to connect to a realtime session, give it a short-lived, scoped token produced by your backend.

This distinction matters even more when the assistant can trigger commerce actions. If a frontend can call “create avatar,” “start session,” or “change voice,” then an attacker who discovers your public JavaScript bundle can often do the same. Security is not about hiding endpoints; it is about constraining authority.

Use API keys only from trusted server code

API keys should authenticate your application to the realtime avatar platform. They are appropriate for:

• creating or updating avatars,

• starting or managing sessions,

• reading usage and operational data,

• provisioning backend-owned resources.

They are not appropriate for client-side rendering, direct browser fetches, or anything a user can inspect in dev tools.

A typical backend pattern is: receive a browser request, validate the user, then call the platform from server code using your API key. For example, in Python:

import os

avatar = resp.json()

import os

avatar = resp.json()

import os

avatar = resp.json()

The exact request fields depend on the API surface, so treat this as illustrative and verify payload shape in the docs. The important part is the deployment model: keep the key in server environment variables or a secrets manager, rotate it, and scope access so only the backend runtime can read it.

A practical hardening checklist:

• Use separate keys per environment: dev, staging, prod.

• Rotate keys periodically and immediately on leak suspicion.

• Log key usage metadata, not the key itself.

• Do not reuse the same key across unrelated services.

JWTs should represent identity, not platform authority

JWTs are useful when your assistant needs to know who the user is, or when the browser needs to present a short-lived proof to your backend. They are not a substitute for API keys, and they should not be treated as a generic “make this secure” token. A JWT can encode claims like:

• subject ID for the user or visitor,

• session ID,

• tenant/store ID,

• role or permission set,

• expiration time.

For a shopping assistant, one useful shape is:

• customer: can ask questions, view personalized recommendations, and submit orders for their own account.

• agent: can view more context, override a recommendation, or assist with support workflows.

• admin: can manage avatars, workflows, and operational settings.

Keep the JWT short-lived. If it is used by the browser, set a small expiration window and refresh it through your backend. If it is used only between your services, still prefer expiration and audience restrictions.

Two practical rules reduce a lot of mistakes:

1. Verify signature, issuer, audience, and expiration on every request.

2. Never trust role claims from the client unless you can verify the token. A JWT is only as good as its signature validation.

Example backend-side authorization check:

def can_manage_sessions(claims: dict) -> bool:
return claims.get("role") in {"agent", "admin"} and claims.get("tenant_id") is not None

def can_manage_sessions(claims: dict) -> bool:
return claims.get("role") in {"agent", "admin"} and claims.get("tenant_id") is not None

def can_manage_sessions(claims: dict) -> bool:
return claims.get("role") in {"agent", "admin"} and claims.get("tenant_id") is not None

That is intentionally boring. Good auth code should be boring. If you need multiple roles, keep the matrix explicit rather than burying logic in UI state.

RBAC should protect actions, not screens

Many teams make the mistake of enforcing roles only in the UI. That is useful for usability, but it is not security. Your backend must enforce RBAC on the actual operations:

• create avatar,

• start or end a session,

• change system instructions,

• read session transcripts,

• access usage or billing data,

• attach commerce tools such as order lookup or inventory search.

For a shopping assistant, this often means splitting privileges between the assistant runtime and the humans who manage it. The assistant’s runtime might be allowed to call product search and cart APIs, but not to issue refunds or change tax settings. A support agent might see customer context but not administrative logs. An admin might manage avatars and keys but not impersonate customers.

Implement RBAC as a server-side policy layer. A simple pattern is:

POLICIES = {

return role in POLICIES.get(action, set())

POLICIES = {

return role in POLICIES.get(action, set())

POLICIES = {

return role in POLICIES.get(action, set())

That style is easy to audit and easy to test. It also makes it obvious when a new endpoint needs an auth decision. If you are building a multi-tenant shopping product, add tenant checks alongside role checks. RBAC answers “can this role do this action,” while tenancy answers “can this principal do it for this customer/store.” You need both.

Session design for realtime assistants

Realtime avatars and voice agents introduce a specific auth problem: the browser often needs to connect to a live media session quickly, but you still want to keep secrets off the client. The correct pattern is usually:

1. User authenticates to your app.

2. Your backend validates the user and decides which role/session they get.

3. Your backend asks the realtime platform to create or authorize the session.

4. Your backend returns only the minimum token or session artifact needed by the browser.

That token should be scoped to the session, not the whole platform. If it is stolen, the blast radius should be one short-lived conversation, not your entire account.

For commerce assistants, also consider the difference between conversational identity and transactional identity. The avatar may be allowed to discuss cart contents, but checkout, refunds, and account changes should require either:

• a backend-verified user session,

• step-up authentication, or

• human confirmation through a separate channel.

This is especially important in voice agents, because conversational cues are easy to over-trust. A person sounding confident over a live avatar is not the same as a properly authenticated user.

Where Protoface fits

Protoface is useful here because it gives you a clean separation between trusted server-side control and untrusted client-side presentation. The REST API is for backend use with API keys, so your application can create avatars and manage sessions without exposing credentials to the browser. If you are integrating a voice agent, the LiveKit plugin in the plugin repository is the right place to add a synchronized face to an existing realtime agent without changing your auth model.

A minimal Python-side integration pattern looks like this:

from protoface import ProtofaceClient

)

from protoface import ProtofaceClient

)

from protoface import ProtofaceClient

)

Treat the fields above as illustrative; use the documented SDK methods and payloads from the docs. The architectural point is what matters: your backend decides which avatar/session to create, then hands the browser only a scoped artifact for realtime connection.

If you prefer a drop-in browser embed for a customer-facing experience, the customer-managed iframe model is even simpler from a security perspective because the API key never reaches the browser. That is a good fit when you want a contained interactive avatar on a website and do not need the frontend to coordinate platform calls directly.

Common failure modes

• Putting API keys in frontend code: the fastest way to create a permanent incident.

• Using JWTs without verification: decoding is not validating.

• Confusing UI gating with authorization: hiding a button does not protect the backend.

• Over-scoping session tokens: a token that can do everything is just a key with worse ergonomics.

• Ignoring tenant boundaries: role checks without tenant checks often leak data across customers.

Also remember the realtime-specific failure mode: once an avatar session is live, it may have access to tools, transcripts, or backend events. Make sure those tool calls are themselves authenticated and authorized. The session transport being secure does not automatically make every downstream action safe.

Conclusion

The secure pattern for a realtime AI shopping assistant is straightforward: keep platform API keys on the backend, use JWTs for short-lived identity and session context, and enforce RBAC on every meaningful action. Do not let the browser directly own privileged platform operations. Scope tokens tightly, separate roles from tenants, and treat realtime media sessions as another authenticated surface, not a special exception.

If you are wiring this up with avatars or a voice agent, start with the docs at docs.protoface.com, then wire the backend first and the UI second. That order usually prevents the security mistakes that are easiest to make under deadline pressure.