Streaming Tokens Without Layout Thrash

The mismatch: LLMs emit 30–80 tokens/second. Displays refresh at 60fps (16ms/frame). Without buffering, you're forcing 30–80 DOM mutations/second when the display can only show 60 frames/second. The excess mutations are wasted CPU that never produce visible output — they happen between frames and are immediately overwritten by the next mutation.

The word-boundary variant flushes at word boundaries (space or punctuation) instead of frame boundaries — this makes streaming feel more natural, since full words appear at once rather than character by character:

let wordBuffer = '';
let rafId = null;

for await (const chunk of stream) {
  wordBuffer += chunk;
  // Flush when we have a complete word (whitespace or punctuation)
  if (/[\s.,!?;:]$/.test(wordBuffer) && !rafId) {
    const toFlush = wordBuffer;
    wordBuffer = '';
    rafId = requestAnimationFrame(() => {
      setContent(prev => prev + toFlush);
      rafId = null;
    });
  }
}
// Flush remainder at stream end
if (wordBuffer) setContent(prev => prev + wordBuffer);

The technical cause: each call to set textContent invalidates the element's layout box. The browser schedules a style recalculation and potential reflow. At 60 mutations/second, that's 60 scheduled reflows per second in a 16ms/frame budget — consuming effectively all available CPU for layout work, leaving nothing for user input handling, scroll, or other animations. On a 4-core machine this is survivable; on a 2-core phone, it jank-locks the thread. The frame rate collapses from 60fps to under 20fps during streaming.

Implementation changes: the critical detail is the if (!rafId) guard — only one rAF is queued at a time, regardless of how many tokens arrive between frames. Without the guard, you queue one rAF per token and recreate the original problem. The buffer itself is a module-scoped string variable (or a React ref), not state — writing to a ref doesn't trigger a re-render, which is the point. The rAF callback is the only place that writes to state, gating re-renders to 60fps maximum.

Open DevTools → Performance and record while each mode streams a 150-token response. In unbuffered mode, you'll see 150 "Recalculate Style" tasks. In buffered mode, about 12 (one per frame at 60fps for a 2-second stream).

The fetch + ReadableStream pattern for authenticated streaming:

const res = await fetch('/api/stream', {
  method: 'POST',
  headers: { 'Authorization': `Bearer ${token}` },
  body: JSON.stringify({ prompt }),
});
const reader = res.body.getReader();
const decoder = new TextDecoder();

let buffer = '', rafId = null;
const flush = (setContent) => {
  setContent(prev => prev + buffer);
  buffer = ''; rafId = null;
};

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  // Parse SSE manually: lines starting with "data: "
  const text = decoder.decode(value);
  for (const line of text.split('\n')) {
    if (line.startsWith('data: ')) {
      const chunk = JSON.parse(line.slice(6))?.delta ?? '';
      buffer += chunk;
      if (!rafId) rafId = requestAnimationFrame(() => flush(setContent));
    }
  }
}

Markdown consideration: streaming markdown mid-parse can produce invalid intermediate states (e.g., a half-rendered code fence). Use a streaming markdown parser like marked with incremental input, or buffer until a complete "block" (double newline) before rendering markdown — rendering raw text until the stream ends, then switching to rendered HTML, is often the simplest correct approach.