What LLM UX Design Actually Means
LLM UX design means designing the visible states of a system that is unpredictable, streams its answer, and can call tools on its own. Do it well and the user always knows what the system is doing. They trust what it shows, and they can stop it at any moment.
Search this phrase and most results teach you to use an LLM as a tool: prompting Figma, generating copy, speeding up your process. That's not this post. This post covers designing the UX of a product built on an LLM, the chat surface, tool calls, an agent acting on a user's behalf.
For the wider framework behind this, unpredictable output, trust calibration, hallucination handling, see the AI UX design framework. This post stays narrow: eight concrete surface mechanics, closing with a table you can build against.
Those eight are streaming reveal, tool call transparency, preview before execute, stop and steer, perceived latency, citations, memory visibility, and structured output rendering.
The Mental Model: Two State Machines on One Surface
Every LLM or agent interface is really two state machines rendered in one place. Miss one and the user reads the gap as a bug.
The first is the message state machine: queued, thinking, streaming, complete, error, stopped.
The second is the tool call state machine, nested inside a message. In code this is often a literal field, part.state, moving through input streaming, input available, approval requested, output available, or output error.
Keep both visible at all times. A model sitting in "thinking" with no signal for several seconds is the most common failure I see in agent products.
Once both machines are legible, the first real decision is what to do with the stream itself.
Step 1: Stream Prose, Buffer Structured Output
Stream natural language prose token by token. Buffer structured output, JSON, tables, code, and render it as one complete block.
The reason is visual. A sentence assembling word by word reads as thinking. A table with a half built row flashing reads as broken.
Under the hood these are two different techniques. Prose comes off an SSE stream into a debounced partial markdown renderer, so an unclosed bold tag never flashes on screen. Structured data waits for a complete, parseable object before it renders.
Handle scrolling the same way. Follow the bottom while new tokens arrive. Pause when the user scrolls up. Offer a jump to latest control so they can rejoin easily.
Step 2: Make Tool Calls and Agent Actions Legible
A tool call is where trust is actually won or lost.
Translate the machine call into plain language. "Executing function 402" means nothing to a user. "Searching your calendar for open slots" tells them what is happening.
Render every tool call through three visible states: started, running, and returned or failed. Each one needs a line in the interface.
For anything that writes or acts, preview it before it executes: the action, the target, and the expected result. Let the user confirm before anything irreversible runs.
Treat autonomy as a dial, not a switch. Four settings cover most products: suggest only, propose a plan, confirm each step, or run within set limits. Start new capabilities at a low setting and widen them as trust builds.
When a call fails, give a three part error: what failed, why, and what to do next. "Could not book that slot, try one of these three instead" beats a raw error code.
On an agent product I built that files support tickets, every ticket shows as an editable draft card with one confirm button before it touches the real queue.
Step 3: Give Users Stop, Steer, and Undo
A visible Stop button only works if it triggers real cancellation. Pass an AbortSignal into the request and cancel the stream server side.
Three controls matter here. Stop halts the task now. Steer redirects a task mid run without starting over. Undo reverses something the agent already finished.
A Stop button that only hides the UI while the model keeps generating behind it is worse than no button at all. The user learns your controls lie, and that distrust spreads to every other control in the product.
Undo matters most for agents that write to real systems. Pair every irreversible action, sending an email, charging a card, deleting a record, with a reviewable audit entry you can find later.
Step 4: Design for Latency, Not Raw Speed
Perceived latency is a design decision, not just a backend number. The metric that matters most is time to first token, TTFT, how long the user waits before anything appears.
Use a simple framework for the wait: stream tokens when you can, the default for chat prose. Show a skeleton when you know the response shape. Save a plain spinner for short, opaque waits, as a last resort.
Fill the dead air before the first token with an honest status label. "Reading the document" or "Searching your calendar" beats a generic "Loading."
Step 5: Show Sources and What the Model Remembers
If your product retrieves information before answering, citations make the answer checkable, not just polished.
Give every retrieved answer three citation layers: an inline marker at the exact claim it supports, a hover or tap preview of the source, and a full reference list under the answer.
Memory needs the same visibility. If the model carries context across turns, files, a prior conversation, facts about the user, show what it holds. A small panel listing a few remembered facts builds more trust than any disclaimer copy.
Let users edit or remove what is remembered. A model that carries a wrong fact for weeks is worse than one that asks again.
Step 6: Render Structured Output as Components, Not Markdown
Whether a response becomes markdown or a real component is a design decision, not whatever shape the model happens to output.
When the data has a known shape, a comparison, a set of options, a form, render it as a table, cards, or input controls. A comparison table should never arrive as a markdown blob the user parses by eye.
Reserve plain markdown for genuinely open ended prose: an explanation, a draft, a summary. Anything a user needs to scan, compare, or act on earns a real component instead.
Three pricing plans in a markdown table are readable. The same plans rendered as selectable cards with a real button are usable. That gap is where an AI product stops feeling like a chat log and starts feeling like software, the same discipline behind a clean Figma to code workflow.
The LLM UX Reference Table: Behavior, Risk, Pattern
Here is the whole post as one table. Match your product's behavior to the risk it creates, then build the pattern in the third column.
| Behavior | UX risk | Pattern | |---|---|---| | Token streaming | Looks frozen before the first token | Stream tokens plus an honest status label | | Tool call | Feels like an opaque black box action | Plain language state for each call: started, running, returned | | Agent write action | Irreversible mistake, no recourse | Preview before execute, confirm, then undo | | Long generation | User waits with no sense of progress | Skeleton tied to time to first token, never a bare spinner | | Retrieved (RAG) answer | Unverifiable, possibly hallucinated claim | Inline citation marker plus a full reference list | | Structured data in reply | Broken looking markup assembling on screen | Buffer the data, then render as a real component | | Long running task | No way to redirect or exit | Visible Stop wired to real cancellation, plus Steer | | Carried memory or context | Feels like the system is guessing about you | A visible, editable panel of what it remembers |
That table is self contained on purpose. Use it as a checklist before your next agent feature ships.
If you are deciding whether to build this in house or bring in outside help, choosing an AI design agency is worth reading first.
I help teams design and ship this kind of interface: message states, tool call transparency, and agent controls that hold up past the demo. If that's the problem in front of you, AI product design is where I can help.