Your Next.js AI streaming route keeps billing you after the user leaves

An AI chat endpoint feels finished the moment tokens start streaming into the browser. Then your first real bill arrives, and the token count is higher than what your users actually read. The usual reason: responses that keep generating after the user has already closed the tab.

Where the money leaks

In the Next.js App Router, a streaming route opens a connection to the model and pipes tokens back to the browser. If the user navigates away or closes the tab mid-response, the browser drops its connection — but the upstream request to the model keeps generating to completion. You pay for every token it produces, and nobody ever sees them. On long answers with an impatient audience, that waste adds up fast.

The fix: stop generating when the browser disconnects

When you build the SSE stream yourself with a ReadableStream, its cancel() callback fires the moment the client connection drops. That's your hook to abort the upstream model request instead of letting it run to completion:

export function POST(req: Request) {
  const upstream = anthropic.messages.stream({ model, max_tokens, messages });

  const body = new ReadableStream({
    async start(controller) {
      for await (const event of upstream) controller.enqueue(encode(event));
      controller.close();
    },
    cancel() {
      // The browser went away — stop generating (and paying).
      upstream.abort();
    },
  });

  return new Response(body, {
    headers: {
      "content-type": "text/event-stream",
      "cache-control": "no-cache, no-transform",
    },
  });
}

That one method turns "streaming works in the demo" into "streaming doesn't leak money in production."

Three things that bite you next

• "no-transform" matters. Some proxies buffer SSE, so your "stream" arrives as a single blob at the very end. The Cache-Control no-transform hint tells them to leave it alone.
• Raise the route's max duration. Long generations outlive the platform's default timeout and get cut off mid-answer; bump maxDuration on the route.
• Meter what you spend. Cancelling saves money you can't see unless you measure it. Record input, output, and cache tokens per request, priced per model, so the savings show up in your numbers instead of your imagination.

Why it belongs in the foundation

None of this is hard. But it's the kind of thing you only discover you needed after it has already cost you — which is exactly why it should live in the foundation of an AI app, not the backlog.

I package this pattern — along with per-request cost metering, typed tool use, versioned prompts with CI evals, and plan-gated quotas — into a Next.js 16 and Claude starter kit called Shipwright. The live demo shows the exact cost of every request.