Introduction

If you are building a realtime AI avatar web app, you are really building two systems at once: a web application and a low-latency media pipeline. The app has to authenticate users, create sessions, manage state, and expose APIs. The media pipeline has to move audio and video with minimal delay, usually over WebRTC or a similar streaming transport, while keeping speech, lip motion, and conversation state synchronized.

The main question is not “Next.js or Express?” in the abstract. It is: where should each responsibility live so you do not overcomplicate the parts that need to stay fast, and do not underbuild the parts that need product logic? By the end of this post, you should have a practical decision framework, know how to wire the backend for a realtime avatar flow, and understand where a platform like Protoface fits when you want to avoid building the media layer yourself.

What Next.js is good at, and what Express is good at

Next.js and Express both run on Node.js, but they sit at different layers of the stack.

Next.js is a full-stack React framework. It gives you routing, server rendering, server actions, API routes, and a good default path for shipping product UI quickly. If your app is mostly a web product with a dashboard, a login flow, a settings page, and a handful of backend endpoints, Next.js is often enough.

Express is a minimal HTTP server framework. It gives you routing and middleware, and little else. That is not a weakness when you need tight control over request handling, long-lived connections, custom middleware, or a backend that may eventually split from the frontend entirely.

For realtime avatar apps, this distinction matters because the web UI and the media/control plane usually have different requirements:

• The UI benefits from co-located frontend and backend code.

• The media/session backend benefits from explicit lifecycle control and a cleaner separation from rendering concerns.

• Any endpoint that issues credentials, creates sessions, or coordinates realtime state should be treated like infrastructure, not just app glue.

Where Next.js fits well

Next.js is a strong choice if your product is primarily a browser app and the avatar is one feature among others. It is especially reasonable when you need:

• A marketing site plus authenticated app in one repo.

• Server-side rendering for session-specific pages.

• API routes for lightweight backend operations.

• Fast iteration with a small team.

For example, you might use a Next.js route handler to mint a short-lived token, create a realtime session, and send the client the connection data it needs. That keeps the browser from ever seeing your long-lived secret key.

import { NextResponse } from 'next/server';

}

import { NextResponse } from 'next/server';

}

import { NextResponse } from 'next/server';

}

The important pattern here is not the framework; it is the trust boundary. The browser should get only what it needs for the current session, and nothing that can be reused as a long-lived credential.

Where Express fits better

Express is usually the better choice when the backend is the product, or when the realtime part has operational complexity that should stay separate from your frontend deployment. In practice, that means:

• You expect custom auth, webhooks, queues, or background jobs around avatar sessions.

• You need stable server behavior for long-lived connections and service orchestration.

• You want the frontend and backend deployed independently.

• You are integrating multiple realtime providers and need a clean service boundary.

Express also tends to be easier to reason about when your app is mostly API-first. A dedicated backend can handle session creation, usage accounting, avatar configuration, and any provider-specific callbacks without mixing that logic into React route handlers.

A minimal Express route for session creation looks similar to the Next.js example, but the deployment story is different: the API can run as its own service, which is often better once the app grows beyond a single product surface.

import express from 'express';

});

import express from 'express';

});

import express from 'express';

});

If you expect the backend to grow into a serious control plane, Express gives you fewer assumptions and less framework coupling. That matters when you are debugging realtime issues under load.

Realtime avatar apps are mostly about latency and trust boundaries

For avatar experiences, the frontend/backend framework choice is secondary to three technical constraints:

1. Latency budget: speech-to-response and lip sync both degrade quickly when extra hops are added.

2. State synchronization: the avatar must reflect the current turn, speaking state, and interruptions correctly.

3. Credential handling: API keys should never be exposed in the browser unless the product is explicitly designed for that and the token is scoped down appropriately.

This is why people often get stuck when they try to build avatar streaming directly inside a generic web framework. The framework is not the hard part; the realtime media plumbing is.

In a typical voice-agent architecture, the browser or app sends audio to an agent backend, the agent produces a response, and the avatar layer renders synchronized video that matches the agent’s speech. That may involve WebRTC, a managed media SDK, or a streaming service. Once you add barge-in, interruptions, and session recovery, the backend stops looking like “just another API.”

How to choose in practice

Use this rule of thumb:

• Choose Next.js if your team wants one codebase for UI and light backend logic, and the avatar is one feature inside a broader web product.

• Choose Express if you want a dedicated backend service, expect complex session orchestration, or already have a separate frontend.

• Use neither as the media layer if you can avoid it. Keep your web framework for product logic, not realtime video transport.

That last point is the one that saves teams the most time. Realtime avatar systems are easiest to maintain when the application layer and the avatar/media layer are decoupled. Your framework should create and authorize sessions, not reinvent streaming.

Where Protoface fits

This is exactly the division of responsibility the platform is designed for. You can keep your web app in Next.js or Express and delegate the avatar layer to a purpose-built service. For a voice-agent stack, the most direct integration is the Pipecat integration or the LiveKit agent plugin, depending on your agent architecture. In both cases, your application code stays focused on orchestration while the avatar layer handles synchronized talking video.

If you are working from Python, the SDK and REST API are the cleanest way to create avatars and sessions from backend code. The exact request fields are documented in the docs, but the shape of the integration is straightforward:

import os

session = resp.json()

import os

session = resp.json()

import os

session = resp.json()

If your use case is a browser-embedded avatar on a customer-facing site, the customer-managed iframe approach avoids exposing backend secrets in the browser and keeps the integration bounded by origin allowlists and rate limits. That is often a better fit than trying to push avatar control directly into client-side React state.

Common gotchas

A few mistakes show up repeatedly in realtime avatar projects:

• Putting API keys in client code. Even with obfuscation, this is not acceptable for a long-lived secret.

• Mixing UI rendering with session orchestration. Keep avatar/session creation behind a server boundary.

• Assuming “realtime” means “WebSocket is enough”. It usually is not; the media path still needs careful timing, buffering, and state management.

• Ignoring interruption behavior. Barge-in and turn taking matter as much as the initial video render.

Also, do not over-optimize framework choice before you know your deployment shape. A clean Next.js implementation can carry you surprisingly far. But if you find yourself writing a lot of custom server logic around sessions, callbacks, or realtime coordination, that is usually the point to move the backend into Express or another dedicated service.

Conclusion

For a realtime AI avatar web app, Next.js is usually the better starting point for product-centric teams, and Express is usually the better long-term home for a dedicated control plane. The key is to keep the framework responsible for app logic, not media transport. Create sessions on the server, keep API keys out of the browser, and let a purpose-built avatar layer handle synchronized video.

If you want to see how the pieces fit together in detail, start with the docs and the relevant quickstart or integration repo. From there, you can choose the right framework boundary for your product instead of forcing realtime media into whatever your frontend happens to use.