# Streaming

Stream a reply token by token from an agent, or read typed chunks straight from an `LLMClient`.

| Call | Tools | Returns |
|---|---|---|
| `agent.streamChat(message)` | Not offered to the model | `Stream<String>` of text tokens |
| `agent.streamChatWithTools(message, handler)` | Run by the agent, up to 10 rounds | The full reply; chunks go to the handler |
| `llm.streamChatWithSpec(request)` | Not run | `Stream<ChatChunk>` |
| `llm.streamChatWithHandler(request, handler)` | Not run | Nothing; chunks go to the handler |

For lifecycle events such as run start, tool start and tool end, use `agent.chatWithEvents`. See [Events](events.md).

<!-- java-contract: src/main/java/com/example/tnsai/docs/StreamingExample.java -->
```java
package com.example.tnsai.docs;

import com.tnsai.agents.Agent;
import com.tnsai.llm.ChatChunk;
import com.tnsai.llm.ChatRequest;
import com.tnsai.llm.LLMClient;

public final class StreamingExample {
    private StreamingExample() {}

    // Text only. The agent's tools are not offered to the model.
    static void tokens(Agent agent) {
        agent.streamChat("Explain relativity").forEach(System.out::print);
    }

    // The agent runs requested tools and streams again, up to 10 rounds.
    static String withTools(Agent agent) {
        return agent.streamChatWithTools("Find the TnsAI release notes", chunk -> {
            if (chunk.isContent()) {
                System.out.print(chunk.getContent());
            } else if (chunk.isToolCall()) {
                chunk.getToolCall().ifPresent(call -> System.out.println("\n[tool] " + call.getName()));
            }
        });
    }

    // Straight from a client: typed chunks, no agent, no tool execution.
    static void chunks(LLMClient llm) {
        ChatRequest request = ChatRequest.builder().message("Tell me a story").build();
        llm.streamChatWithSpec(request).forEach(chunk -> {
            switch (chunk.getType()) {
                case START -> System.out.println("model: " + chunk.getModel().orElse("?"));
                case CONTENT -> System.out.print(chunk.getContent());
                case TOOL_CALL -> System.out.println("\n[tool call] " + chunk.getContent());
                case DONE -> System.out.println("\nfinish: " + chunk.getFinishReason().orElse(ChatChunk.FinishReason.STOP));
                case ERROR -> System.err.println("error: " + chunk.getContent());
            }
        });
    }
}
```

## Tool rounds

`streamChatWithTools` streams one reply. If the model asked for tools, the agent runs them, adds the results to memory and streams again. It stops at a reply without tool calls, or after 10 rounds with a warning in the log.

The handler gets the first `START` chunk, every `CONTENT` and `TOOL_CALL` chunk, and one `DONE` chunk with `STOP` at the end. It does not get `ERROR` chunks: a failed model call ends the round with no text, and the method returns an empty string.

## Client streaming

`streamChatWithSpec` and `streamChatWithHandler` only report what the model sends. A `TOOL_CALL` chunk carries the call; running it is up to you.

With tools in the request, `streamChatWithHandler` does not stream: it makes one `chat` call and sends the reply as one `CONTENT` chunk, then one `TOOL_CALL` chunk per call, then `DONE` with `TOOL_CALLS` or `STOP`. It turns any exception into an `ERROR` chunk, except an `LLMCapabilityException`, which it throws.

The default `streamChatWithSpec` wraps the client's text stream in `START`, `CONTENT` and `DONE` chunks. A provider may override it with richer chunks.

### `ChatChunk`

| Type | Carries |
|---|---|
| `START` | `getModel()`, `getRequestId()` |
| `CONTENT` | `getContent()`, and `getTokenCount()` and `getChunkIndex()` when known |
| `TOOL_CALL` | `getToolCall()`, and `getContent()` as `name(arguments)` |
| `DONE` | `getFinishReason()`: `STOP`, `LENGTH`, `TOOL_CALLS`, `CONTENT_FILTER` or `ERROR` |
| `ERROR` | `getContent()`, the error message |

`isStart()`, `isContent()`, `isToolCall()`, `isDone()` and `isError()` test the type. The static factories `start`, `content`, `toolCall`, `done` and `error` build chunks for tests or custom clients.

## Per-call retrieval context

The streaming, event and action calls each have an overload that takes a `ChatRetrievalContext`. It scopes one call: tenant and session, run id, cancellation token, deadline and metadata filters. Fields you leave empty get the agent's tenant, session and name; fields you set are kept. `chatWithEvents(message, sessionId)` takes no context.

<!-- java-contract: src/main/java/com/example/tnsai/docs/ActionRetrievalContextExample.java -->
```java
package com.example.tnsai.docs;

import com.tnsai.actions.model.ActionRequest;
import com.tnsai.agents.Agent;
import com.tnsai.cancellation.CancellationToken;
import com.tnsai.rag.ChatRetrievalContext;

import java.time.Duration;
import java.util.Map;

public final class ActionRetrievalContextExample {

    private ActionRetrievalContextExample() {}

    public static void dispatch(Agent agent) {
        CancellationToken cancellation = CancellationToken.create();
        ChatRetrievalContext context = ChatRetrievalContext.builder()
            .scope("tenant-a", "conversation-42")
            .runId("run-7")
            .simulationState("review")
            .currentEpisode("episode-3")
            .sourceRevision("kb-2026-08-12")
            .cancellationToken(cancellation)
            .timeout(Duration.ofSeconds(3))
            .metadataFilters(Map.of("tier", "approved"))
            .build();

        agent.streamChatWithTools("Summarize the evidence", chunk -> {}, context);
        agent.streamChatWithRetrievedContext(
            "Fenced outbound request", "Original user message", chunk -> {}, context
        );
        agent.chatWithEvents("Summarize the evidence", event -> {}, context);

        agent.executeAction(
            "answer", Map.of("question", "What changed?"), context
        );
        agent.executeAction(
            ActionRequest.of("answer", Map.of("question", "What changed?")),
            context
        );
        agent.executeActionOnRole(
            "researcher", "answer", Map.of("question", "What changed?"), context
        );
        agent.executeActionOnRole(
            "researcher",
            ActionRequest.of("answer", Map.of("question", "What changed?")),
            context
        );
    }
}
```

Cancellation is cooperative. A retriever that blocks on its own backend has to apply the deadline itself. For what the context does to knowledge retrieval, see [Ground an action](../tutorials/grounded-action.md).

## `AsyncAgent`

`com.tnsai.agents.async.AsyncAgent` is an interface with no implementation in the framework. To use it, implement it yourself.

## Related

- [Events](events.md): typed lifecycle events for a chat turn.
- [Actions and tools](actions-and-tools.md): what the model can call.
- [Ground an action](../tutorials/grounded-action.md): knowledge retrieval for one call.