Introduction

Janky animation in a realtime avatar UI usually isn’t a “graphics” problem. It’s a systems problem: audio arrives late, video frames are decoded unevenly, React rerenders at the wrong time, and the browser’s rendering pipeline gets asked to do too much. The result is what users notice immediately: lip sync drifting, faces freezing for a beat, transitions popping, or a talking head that feels oddly detached from the voice agent behind it.

This post is about making those failures predictable and fixable. By the end, you should be able to reason about where jitter comes from in a realtime avatar stack, keep React from amplifying it, and choose a rendering strategy that stays smooth under network and device variability.

Start with the actual frame pipeline

For a talking avatar, you’re usually dealing with at least three clocks:

• Audio generation / transport — the model or voice service produces speech chunks, often over WebRTC or a similar low-latency stream.

• Video synthesis / transport — the avatar service produces lip-synced frames or a live video stream that must stay aligned to the audio.

• UI rendering — React renders controls, state, overlays, and the video element or iframe container.

“Jank” happens when these clocks drift relative to each other. The biggest mistake is treating the avatar like a normal animated component. It is not. A realtime avatar is closer to a media player with application state wrapped around it.

Keep React out of the hot path

React is good at describing UI state, but it is not the place to drive per-frame animation logic for a live avatar. If you bind every session update, speaking-state change, or network event directly to component state, you can easily create unnecessary rerenders that block the main thread right when the browser needs to decode and paint video.

Practical rules:

• Store fast-changing transport state in refs or an external store, not in deeply nested component state.

• Only promote state into React when the UI actually needs to change: connection status, mic permission, selected avatar, or a coarse speaking indicator.

• Avoid rerendering the video element itself unless the media source changes.

• Debounce “chatty” events like partial transcripts, network stats, or frame telemetry.

In other words: let the media pipeline run, and let React observe it at a low enough frequency to stay out of the way.

Prefer stable media surfaces over re-creating DOM

A common source of jank is accidentally tearing down and recreating the video surface. In React, that can happen when a parent component changes keys, conditionally swaps components, or recreates callback props in a way that causes a full subtree reset. If the avatar’s video element gets replaced, you lose decoder state and often induce a visible hitch.

For direct media playback, keep the surface stable:

import { useEffect, useRef } from "react";

}

import { useEffect, useRef } from "react";

}

import { useEffect, useRef } from "react";

}

The important part is not the snippet itself; it’s the principle. Attach the stream once, keep the element mounted, and update surrounding UI independently.

Use explicit loading and buffering states

Realtime avatars often look “broken” for the first second or two even when nothing is wrong. The stream is warming up, audio is buffering, or the agent hasn’t produced the first synthesized face yet. If you render the final avatar shell immediately, users interpret the blank or frozen surface as lag.

Instead, model the UI as a small state machine:

1. Idle — no session yet.

2. Connecting — session is being created and media is negotiating.

3. Warming up — transport is live, but the first meaningful frame hasn’t arrived.

4. Live — avatar is actively speaking or ready to speak.

5. Degraded — media is available but behind or dropped frames are detected.

That extra granularity helps in two ways. First, you can show a sane placeholder instead of a frozen face. Second, you can delay cosmetic transitions until the stream is stable. A subtle fade-in after the first decoded frame is usually better than animating the avatar container on every status update.

Don’t animate what the browser already animates

Browser media elements already have their own timing and paint behavior. If you layer CSS transitions, transforms, and opacity animations on top of a live video element, you can make the problem worse. The expensive path is not the avatar frame itself; it is often the surrounding DOM churn and compositor work.

Use animations sparingly:

• Animate container chrome, not the video surface, unless you have a clear reason.

• Prefer transform and opacity over layout-triggering properties like width, height, top, and left.

• Keep expensive shadows, blurs, and filters off the live video area.

• Don’t animate every speaking pulse if the stream already conveys motion.

If you need to show “the agent is speaking,” a small, isolated indicator is enough. The avatar itself should be the visual focus, not a constantly moving stack of wrappers.

Handle lag as a media problem, not a UI mystery

When lip sync looks off, the first question is whether the problem is in the source or in the rendering path. A few useful checks:

• Audio lead/lag: if the voice sounds correct but the mouth is behind, your video frames are delayed relative to audio.

• Frame cadence: if motion stutters, the issue may be decode or paint jank rather than network loss.

• Main-thread pressure: if the UI freezes during transcript updates, your app logic is competing with paint.

• Session resets: if the avatar visibly restarts, the component tree may be remounting the media surface.

At the implementation level, measure the gap between media events and UI updates. If you’re seeing unstable render timings in the browser, start by reducing rerender frequency and removing work from the avatar subtree. Then look at transport conditions and service latency.

Where Protoface fits: drop the avatar into the media layer, not the React tree

For teams building on LiveKit, the cleanest way to avoid React-induced jank is to keep the avatar in the agent/media layer and let the app treat it as a stable session resource. The quickstart examples are useful for seeing the broader pattern, but the relevant piece here is the LiveKit Agents plugin: you add the avatar to the voice agent, and the agent gets a synchronized talking video face without your React app having to orchestrate per-frame behavior.

A minimal plugin setup looks like this in spirit:

# illustrative only; exact config fields are in the docs

await avatar.attach_to_agent(ctx.agent)

# illustrative only; exact config fields are in the docs

await avatar.attach_to_agent(ctx.agent)

# illustrative only; exact config fields are in the docs

await avatar.attach_to_agent(ctx.agent)

The advantage is architectural: React owns the surrounding product UI, while the avatar stream stays on the realtime side where it belongs. That separation reduces rerenders, preserves decoder state, and makes it easier to treat motion issues as media issues instead of frontend mysteries. If you are wiring the agent server side, the plugin repo and docs are the right place to confirm exact setup details: docs.protoface.com and the plugin examples in the GitHub org.

Be careful with session lifecycle and embeds

If you’re embedding an avatar on a website, lifecycle mistakes show up as jank very quickly. Recreating the iframe, changing its URL on every prop update, or forcing full reloads for minor config changes can make the avatar look unstable even when the backend is fine.

Keep these patterns in mind:

• Create the iframe once and update only when session-level configuration truly changes.

• Don’t expose API keys in the browser; use the customer-managed embed flow.

• Treat per-embed instructions, voice selection, and allowlists as session configuration, not UI state.

• If you need to change behavior mid-conversation, prefer a session update path rather than a hard reload.

This is one place where a managed avatar surface helps: it removes a lot of accidental frontend complexity from the critical path. The browser can focus on layout and controls, while the avatar runtime handles the media session.

A small debugging checklist that actually helps

When an avatar feels janky, use a narrow checklist instead of guessing:

1. Check rerenders: is the avatar subtree rerendering on every transcript token or telemetry event?

2. Check remounts: is the video element or iframe being recreated because a key changed?

3. Check long tasks: are there expensive synchronous updates on the main thread while the stream is active?

4. Check media stability: are frames arriving consistently, or is the source itself stalling?

5. Check loading states: are you showing a deliberate “warming up” state instead of a broken-looking blank surface?

If you fix these in order, you usually get a bigger improvement than by tweaking animation timing curves or adding more CSS transitions.

Conclusion

Janky realtime avatar UIs are usually the result of mixing concerns: media transport, frame decoding, and React state updates all happening in the same critical path. The cure is to make the avatar surface stable, keep React out of per-frame work, and treat buffering and lifecycle as first-class states rather than edge cases.

If you’re building this kind of stack, start by stabilizing the media element or embed, then move avatar orchestration into the agent layer where possible. For implementation details, integration options, and quickstarts, see docs.protoface.com and the examples linked from the GitHub org.