Building AI Features in React

Course Content

Building AI Features in React

5 sections · 21 lessons

Feature 2: A Streaming Summary With a Stop Button

The summary is the feature shoppers notice. It is also where streaming, limits and cancellation all meet. A summary that appears word by word in under a second feels fast, but only if every link in the chain streams. A Stop button feels responsive, but it only saves money if the abort travels all the way to the provider.

This lesson builds the summary route and the React side, and then follows a Stop press through every layer to show where it could silently fail.

Where one Stop press has to travelStop callsabort() onthe controllerThe fetchreader rejectsThe browsercloses theconnectionExpressemits closeon the responseUpstream abortends generationTest the last two steps: a UI that stops while the server generates still pays.
A Stop button only saves money if the abort crosses the BFF and reaches the provider.

The route, part 1: preconditions and cancellation

TypeScript
// server/routes/summary.ts (part 1)import type { Request, Response } from "express";import { SummaryRequest } from "../../shared/schemas";import { getProduct, getTopReviews } from "../data";import { llm } from "../llm";import { SUMMARY_SYSTEM, summaryUserPrompt } from "../prompts";const MAX_WORDS = 80;const SENTINEL = "NOT_ENOUGH_INFO";const wordCount = (s: string) => s.trim().split(/\s+/).filter(Boolean).length;export async function summaryRoute(req: Request, res: Response): Promise<void> {  const input = SummaryRequest.safeParse(req.body);  if (!input.success) { res.status(400).json({ error: "Invalid request" }); return; }  const { productId } = input.data;  const [product, reviews] = await Promise.all([getProduct(productId), getTopReviews(productId, 40)]);  if (!product) { res.status(404).json({ error: "Unknown product" }); return; }  if (reviews.length < 5) { res.status(422).json({ error: "not_enough_reviews" }); return; }  const upstream = new AbortController();  res.on("close", () => upstream.abort()); // Stop, navigation or a closed tab  try {    const outcome = await pipeSummary(res, llm.stream({      tier: "main", system: SUMMARY_SYSTEM, user: summaryUserPrompt(product.name, reviews),      maxTokens: 300, signal: upstream.signal,    }));    if (outcome === "sentinel") { res.status(422).json({ error: "not_enough_reviews" }); return; }    res.end();  } catch (err) {    if (upstream.signal.aborted) return; // the shopper left; there is no one to answer    if (!res.headersSent) res.status(502).json({ error: "Summary unavailable" });    else res.destroy(); // mid-stream failure: cut the connection so the client sees it  } finally {    upstream.abort(); // if generation is still running for any reason, stop it now  }}

The cheap checks come first: a valid body, a known product, at least 5 reviews. None of them costs a model call. Then the route creates its own AbortController for the model call and connects it to the response's close event. When the browser disconnects, for whatever reason, the model call is aborted. The finally block aborts again after every path; aborting a finished call does nothing, and it guarantees no path leaves generation running.

The route, part 2: holding, limiting and writing

TypeScript
// server/routes/summary.ts (part 2)function begin(res: Response, text: string) {  res.setHeader("Content-Type", "text/plain; charset=utf-8");  res.setHeader("Cache-Control", "no-store");  res.write(text); // the first write sends the headers}async function pipeSummary(  res: Response, chunks: AsyncIterable<string>,): Promise<"sent" | "sentinel" | "cut"> {  let text = "";  for await (const chunk of chunks) {    text += chunk;    if (!res.headersSent) {      if (SENTINEL.startsWith(text.trim())) continue; // could still be the sentinel: hold      begin(res, text);    } else if (wordCount(text) > MAX_WORDS) {      res.write(" …");      return "cut"; // leaving the loop ends the model stream    } else {      res.write(chunk);    }  }  if (text.trim() === SENTINEL) return "sentinel";  if (!res.headersSent) begin(res, text); // a very short reply that never left the hold  return "sent";}

This function does three things in one pass. First, it holds the start of the stream while the text so far could still become NOT_ENOUGH_INFO. As soon as the text stops matching, which is usually after the first token, everything held is written at once. The cost is a delay of one or two tokens, about 30 ms, and the benefit is that the sentinel never reaches the shopper and can still become a proper 422 response, because no headers have been sent. Second, it limits: once the text passes 80 words, it writes an ellipsis and returns. Returning from inside for await closes the model stream, and the finally block in part 1 aborts it too. Third, it writes each new chunk as soon as it arrives. Word counting runs on the whole text, not on each chunk, because a chunk can end halfway through a word.

The hook

TypeScript
// src/hooks/useStreamingSummary.ts (first version)import { useCallback, useEffect, useRef, useState } from "react";import { streamText } from "../api/client";type Status = "idle" | "streaming" | "done" | "error" | "cancelled";export function useStreamingSummary(productId: string) {  const [text, setText] = useState("");  const [status, setStatus] = useState<Status>("idle");  const active = useRef<AbortController | null>(null);  const start = useCallback(() => {    active.current?.abort(); // one summary at a time    const controller = new AbortController();    active.current = controller;    setText("");    setStatus("streaming");    streamText("/api/summary", { productId }, (chunk) => {      if (active.current === controller) setText((t) => t + chunk);    }, controller.signal)      .then(() => { if (active.current === controller) setStatus("done"); })      .catch(() => {        if (active.current !== controller) return; // a newer request owns the UI        setStatus(controller.signal.aborted ? "cancelled" : "error");      });  }, [productId]);  const stop = useCallback(() => active.current?.abort(), []);  useEffect(() => {    start();    return () => active.current?.abort();  }, [start]);  return { text, status, start, stop };}

The ref active holds the controller of the request that currently owns the UI. Every callback checks active.current === controller before touching state. Without that check, here is the bug: the shopper switches product, start aborts the old request and begins a new one, and a moment later the old request's catch runs and sets the status to "cancelled" while the new summary is streaming. In development, React 18's Strict Mode runs the effect twice, so this guard also handles the first, immediately aborted request. Thanks to the server's close handler, that doubled request costs almost nothing.

The component

TSX
// src/components/BuyerSummary.tsx (first version)import { useId } from "react";import { useStreamingSummary } from "../hooks/useStreamingSummary";export function BuyerSummary({ productId }: { productId: string }) {  const { text, status, start, stop } = useStreamingSummary(productId);  const titleId = useId();  return (    <section className="buyer-summary" aria-labelledby={titleId}>      <div className="buyer-summary__head">        <h2 id={titleId}>What buyers say</h2>        {status === "streaming" && <button type="button" onClick={stop}>Stop</button>}      </div>      <p className="buyer-summary__text" aria-busy={status === "streaming"}>        {text}        {status === "streaming" && <span className="caret" aria-hidden="true" />}      </p>      {status === "cancelled" && (        <p className="muted">Stopped. <button type="button" onClick={start}>Start again</button></p>      )}      {status === "error" && (        <p className="muted">Summary unavailable. <button type="button" onClick={start}>Try again</button></p>      )}    </section>  );}

The text is rendered as a normal React child, so React escapes it; nothing the model writes can become HTML. There is no aria-live on the streaming paragraph on purpose: a live region would make a screen reader announce every chunk, dozens of times per summary. Section 4 adds a single announcement when the summary is complete.

Following one Stop press

  1. Button — stop() calls abort() on the active controller.
  2. Browser — reader.read() rejects, the fetch is cancelled and the TCP connection to the BFF closes.
  3. Hook — the catch sees signal.aborted and sets the status to "cancelled"; the partial text stays on screen.
  4. BFF — Express emits close on the response, and the route aborts upstream.
  5. SDK — the aborted signal closes the connection to the provider, which stops generating.

Test the last two steps, not just the first three. It is easy to build a Stop button that updates the UI while the server keeps generating to the end, still paying for every token. Add a log line in the close handler and one when the model stream ends, press Stop, and check the timestamps are milliseconds apart.

Check your understanding

0 of 3 answered

1.Why does the server hold the first few characters of the stream before writing anything?

2.The Stop button updates the UI, but provider usage shows every summary running to its full length. What is the most likely gap?

3.A shopper switches from product A to product B while A's summary is streaming. Without the active.current === controller checks, what can go wrong?