TermAl

Feature Brief: Streaming-Aware Markdown Rendering

Status

Active. Lands in the current tree.

MarkdownContent (ui/src/message-cards.tsx) gains an isStreaming prop (default false). When set, an in-flight trailing structural block — pipe-table, fenced code block, or $$ ... $$ math display block — is rendered as plain text in a styled <pre class="markdown-streaming-fragment"> placeholder until the block closes. The settled prefix continues through the existing react-markdown + remark-gfm + remark-math + rehype-katex pipeline. Once the block settles, the next textDelta re-runs the splitter and the placeholder snaps to canonical Markdown rendering.

The gating site (MessageCard in the same file) decides whether a streaming assistant message has Markdown structure worth rendering through this pipeline; if not, it stays on StreamingAssistantTextShell’s plain-text fast path.

Settled callers — history bubbles, source-renderer previews, the rendered Markdown viewer (docs/features/markdown-document-view.md), and rendered Git diff regions — leave isStreaming at its default false and get the existing pipeline unchanged.

Problem

Streaming assistant replies arrive as a sequence of textDelta events. Each current delta carries textStartByte, so the client first verifies that its retained UTF-8 prefix ends exactly where the new chunk begins. A mismatch means an earlier SSE event was lost; the client leaves the last valid draft untouched and hydrates authoritative session text rather than appending across the gap. Once continuity is established, each delta extends the message text by some bytes. CommonMark/GFM rendering is intrinsically structural:

Without intervention, the user sees visibly-broken intermediate shapes flicker on every chunk:

The user-visible result is a transcript that constantly reflows during streaming — distracting and harder to read, especially on slower devices. Tables and math are the most common offenders because the canonical shape is large and the partial shape looks broken.

Solution

A small pure module owns the partial-block detection: ui/src/markdown-streaming-split.ts. Its single export, splitStreamingMarkdownForRendering(markdown), returns { settled, pending }. The cut policy:

  1. Walk lines tracking three structural states:
    • inFence — opened by a or ~~~` line, closed by the next matching marker.
    • inMath — opened by $$ on its own line, closed by the next $$ on its own line.
    • tableStart — any line starting with |; reset on a blank line, a non-pipe text line, or when a fence or $$ block opens (because the candidate pipe lines were re-interpreted as something else, so the table tracking would be a false positive).
  2. Cut at the earliest open-block start. Anything before the cut is settled and safe for react-markdown. Anything after is pending and renders as plain text.
  3. Boundary newlines live at the end of settled so callers can reconstruct the original via plain settled + pending concatenation, regardless of which half is empty.

Settled callers pass isStreaming={false} (the default) and short- circuit the splitter — settled === markdown and pending === "" for any input.

Pipeline Integration

MarkdownContent runs the splitter inside a useMemo keyed on [isStreaming, markdown]. Three downstream concerns key on settledMarkdown rather than the raw markdown:

This keeps an in-flight unclosed fence or $$ block from counting against per-document caps before its real shape is known. It also means the placeholder’s plain text is excluded from line-marker tracking (it carries no data-markdown-line-start attributes).

The pending half is rendered inside <pre className="markdown-streaming-fragment"> (styled in ui/src/styles.css), with aria-busy="true" to signal in-flight content to assistive technology. The styling is deliberately calm — muted background, monospace, slightly reduced opacity — so the “in-flight, not the final shape” signal is clear without being noisy.

MessageCard Gate

StreamingAssistantTextShell is the existing plain-text fast path for streaming assistant replies — used when the message has not yet shown any sign of Markdown structure. MessageCard decides which path to take via hasRenderableStreamingMarkdown(text), which detects:

If any of these match, the path switches to <MarkdownContent isStreaming />. Otherwise it stays on the plain <p> shell, which has the lowest possible streaming cost.

The session pane only enables this fast path for the active turn’s last transcript item when that item is assistant text. Older assistant messages stay on the settled Markdown path while a later user prompt, approval card, or tool request is active, so completed tables do not briefly fall back to streaming placeholders during the next turn.

Test Coverage

Files