Searching an agent's run files

Giving it to a chat

The retriever is a RetrievalEngine.Retriever. A host gives it to a chat as an undeclared binding, a source no @Knowledge declares (see Exact owner and isOwnedBy):

var retriever = IncrementalTextFilesRetriever.builder(runFolder, "Alice")
        .interpreter(hostInterpreter)            // what the host's records mean
        .masker(Masker.hidingOutside(runFolder)) // opt-in; masking is off by default
        .build();
var binding = ChatKnowledgeBinding.undeclared(
        agent, "Alice", "agent-files", 5, retriever); // owner, owner id, source name, topK

What a record is

A host tells the retriever what its records mean with a RecordInterpreter. Its read(file, line, record) returns a Reading: a note, a message the agent was told or said, or a record leftOut. Without an interpreter every record is a note.

• With a host interpreter, records of being told the turn message are not returned, and a message several files record is given once.
• A note several files record is given once with any interpreter.
• Log records at DEBUG and TRACE are left out.

Line format

A host that keeps per-agent .history and .log files writes them with the logback patterns of AgentRecordFormat. The episode state goes under the episode-state MDC key and the conversation under conversation-id. Every line carries a conversation field before the writer's name, - on a line written outside any conversation, so a history line still ends with ] <agent> <message>:

2026-10-21 18:00:00 [running] [conv-1] Alice has started episode 1.
2026-10-21 18:00:00 INFO  [running] [-] Alice chat started

A message of several lines is written with two spaces before each line after its first. RecordParser parses a line only from its start, so an indented line continues the record before it and never starts one: a line in a message that looks like a record, such as one of another conversation, stays in the record it was written in. RecordParser also reads lines written without the conversation field.

Known limit. Only a message's own lines are indented. Lines written before 0.18.6, and the stack trace logback appends after a message, are not, so a line in them that looks like a record of another conversation is read as one.

A reader that splits these lines by position, such as a tail view, has to skip the new field; see Migration.

Episodes and conversations

The request's current episode is the decimal N of episode-N in its ChatRetrievalContext.CURRENT_EPISODE_FILTER metadata filter; any other value is rejected.

• Scope.RUN, the default, searches every episode of the run. Scope.EPISODE searches only the current episode and finds nothing without one. A record of another episode is evidence whatever conversation it was written in, so Scope.RUN suits a single-user host; a host whose run serves several users uses Scope.EPISODE.
• A .history or .log record is evidence only for a request with both a current episode and a conversation id. A request without either gets no such record from any episode; its other files are still searched.
• With both, a record of the current episode is evidence when it was written in the request's conversation or outside any (-).
• A line without the conversation field in the current episode does not enter evidence and is reported with one WARN per file. A line before a file's first record counts as one. A run written before 0.18.6 has no field on any line, so its other episodes are searched and its current episode's records are not.
• Other files are not filtered by conversation.

Files

The run folder is taken by its real path, so a run folder that is itself a symbolic link is followed. Links below it are not followed, and a file whose real path lies outside the run folder is left out with one WARN. A file with a NUL byte in its first 8 KiB is left out.

Masking

Evidence text passes through the builder's Masker before it is indexed, so what the masker hides is neither returned nor searched. A record is masked whole before it is cut into lines and pieces, so a key block over several of its lines is hidden whole. See Masking local paths and credentials for Masker.hidingOutside and PathRedactor.

Known limit. Masking sees one read at a time. A record whose lines arrive in separate reads, such as one still being written when a search reads it, or one across the 1 MiB block a long unread stretch is read in, is masked in parts, so a key block over those lines can be left partly visible.

Cost

The retriever has no file, piece or line cap: its index holds every line of the agent's files in the run. Each search reads only what was appended since the last one, and reads a file again from its start when it shrank, was replaced or was rewritten. A piece of evidence holds at most MAX_PIECE_CHARS (2000) characters of a line, a longer line being cut into pieces, and a result holds at most the request's topK pieces.