# Fan-out and agent pools

Send one message to many agents and combine what comes back with `ScatterGather`. Reuse agents across requests with `LazyAgentPool`. Both live in `tnsai-coordination`.

## Scatter-gather

`ScatterGather` is a static utility. Every method sends the same message to each agent through `agent.chat(...)` on a thread pool of up to 10 threads, and collects an `AgentResponse` per agent.

| Method | Returns | Waits for |
|---|---|---|
| `scatter(message, agents, timeout, unit)` | `ScatterResults` | Every agent, or the timeout. `scatter(message, agents)` uses 60 seconds. |
| `race(message, agents, timeout, unit)` | `Optional<AgentResponse>` | The first successful response. Failed responses do not win. |
| `quorum(message, agents, quorumSize, timeout, unit)` | `ScatterResults` | `quorumSize` successful responses, every agent, or the timeout |
| `gatherWithScore(message, agents, scorer, timeout, unit)` | `Optional<AgentResponse>` | Runs `scatter`, then picks the successful response with the highest `scorer` value |
| `gatherWithConsensus(message, agents, timeout, unit)` | `Optional<AgentResponse>` | Runs `scatter`, then picks the response that the most other responses resemble |
| `gatherAndAggregate(message, agents, aggregator, timeout, unit)` | `Optional<String>` | Runs `scatter`, then asks the `aggregator` agent to merge the successful answers |

<!-- java-contract: src/main/java/com/example/tnsai/docs/FanOutExample.java -->
```java
package com.example.tnsai.docs;

import com.tnsai.agents.Agent;
import com.tnsai.coordination.ScatterGather;
import com.tnsai.coordination.ScatterGather.AgentResponse;
import com.tnsai.coordination.ScatterGather.ScatterResults;

import java.util.List;
import java.util.Optional;
import java.util.concurrent.TimeUnit;

/** Compilation contract for multi-agent/fan-out.md scatter-gather. */
public final class FanOutExample {

    private FanOutExample() {
    }

    static void review(String question, List<Agent> experts, Agent editor) {
        // Ask everyone and wait up to 30 seconds.
        ScatterResults all = ScatterGather.scatter(question, experts, 30, TimeUnit.SECONDS);
        System.out.println("Answered: " + all.successful().size() + "/" + all.totalAgents());
        all.fastest().ifPresent(r -> System.out.println("Fastest: " + r.agentName()));

        // Take the first successful answer.
        Optional<AgentResponse> first = ScatterGather.race(question, experts, 10, TimeUnit.SECONDS);
        first.ifPresent(r -> System.out.println(r.agentName() + ": " + r.response()));

        // Stop waiting once two agents have answered.
        ScatterResults two = ScatterGather.quorum(question, experts, 2, 30, TimeUnit.SECONDS);
        System.out.println("Quorum answers: " + two.responseTexts());

        // Let an editor agent merge the answers.
        Optional<String> merged = ScatterGather.gatherAndAggregate(
            question, experts, editor, 30, TimeUnit.SECONDS);
        merged.ifPresent(System.out::println);
    }
}
```

### Results

`AgentResponse` is a record with `agentId`, `agentName`, `response`, `durationMs`, `success` and `error`. An agent that throws becomes a failed response with the exception message as `error`; an agent still running at the timeout becomes a failed response with the error `Timeout`.

`ScatterResults` holds `responses`, `totalDurationMs` and `totalAgents`, plus these helpers:

| Helper | Returns |
|---|---|
| `successful()` / `failed()` | The responses split by `success` |
| `responseTexts()` | The text of each successful response |
| `fastest()` | The successful response with the lowest `durationMs` |
| `successRate()` | Successful responses divided by `totalAgents` |

`quorum` returns only the responses collected so far, so `responses` can hold fewer entries than `totalAgents`. The agents still running are interrupted.

### Consensus and aggregation

`gatherWithConsensus` compares responses word by word. Two responses count as similar when the Jaccard similarity of their lower-cased word sets is above 0.5. The response similar to the most others wins. It does not compare meaning, so use `gatherWithScore` or an aggregator when answers are phrased differently.

`gatherAndAggregate` returns the single answer as-is when only one agent succeeds. Otherwise it sends the aggregator the original question and each successful answer, labelled with the agent's name, and returns the aggregator's reply. If the aggregator fails, it returns the first successful answer.

## Agent pools

`LazyAgentPool` creates agents through an `AgentFactory` when they are checked out and keeps returned agents idle for reuse. `checkout()` returns an `AgentHandle`. Use it in try-with-resources: `close()` returns the agent to the pool.

<!-- java-contract: src/main/java/com/example/tnsai/docs/AgentPoolExample.java -->
```java
package com.example.tnsai.docs;

import com.tnsai.agents.Agent;
import com.tnsai.coordination.pool.ActivationPolicy;
import com.tnsai.coordination.pool.AgentFactory;
import com.tnsai.coordination.pool.AgentHandle;
import com.tnsai.coordination.pool.LazyAgentPool;
import com.tnsai.coordination.pool.PoolMetrics;
import com.tnsai.coordination.pool.PredictiveActivator;

import java.util.concurrent.TimeUnit;

/** Compilation contract for multi-agent/fan-out.md agent pool. */
public final class AgentPoolExample {

    private AgentPoolExample() {
    }

    static void serve(AgentFactory factory, String task) {
        LazyAgentPool pool = LazyAgentPool.builder()
            .factory(factory)
            .minIdle(2)
            .maxSize(20)
            .idleTimeout(5, TimeUnit.MINUTES)
            .policy(ActivationPolicy.PREWARMED)
            .build();

        PredictiveActivator activator = PredictiveActivator.builder()
            .pool(pool)
            .windowSize(60)        // seconds
            .burstThreshold(5)     // checkouts per window
            .preWarmCount(3)
            .analyzeInterval(10)   // seconds
            .build();
        activator.start();

        try (AgentHandle handle = pool.checkout()) {
            String answer = handle.execute(task);
            Agent agent = handle.getAgent();
            System.out.println(agent.name() + ": " + answer);
        }   // close() returns the agent to the pool

        PoolMetrics metrics = pool.getMetrics();
        System.out.println("Active " + metrics.active() + ", idle " + metrics.idle()
            + ", peak " + metrics.peakActive());

        activator.stop();
        pool.close();
    }
}
```

| Builder method | Default | Meaning |
|---|---|---|
| `factory(AgentFactory)` | Required | Creates agents. `AgentFactory` is a functional interface with `Agent create()`. |
| `minIdle(int)` | 0 | Idle agents the evictor keeps |
| `maxSize(int)` | 10 | Upper limit on active plus idle agents. Must be at least `minIdle`. |
| `idleTimeout(long, TimeUnit)` | 60 seconds | Sets the eviction interval |
| `policy(ActivationPolicy)` | `ON_DEMAND` | When agents are created |
| `listener(PoolListener)` | None | Receives pool events |

`checkout()` throws `IllegalStateException` when the pool is closed, or when `maxSize` agents already exist. In the second case listeners get `onExhausted()` first. `AgentHandle` also has `getAgent()` and `getCheckoutDurationMs()`.

### Activation and eviction

| `ActivationPolicy` | Behavior |
|---|---|
| `ON_DEMAND` | Creates an agent at checkout when none is idle |
| `PREWARMED` | Also creates `minIdle` agents when the pool is built |
| `BURST` | Declared, but the pool treats it like `ON_DEMAND` |

Every half of `idleTimeout` (at least once per second), the pool destroys idle agents above `minIdle`. It does not check how long each agent has been idle, so agents added by `preWarm` are removed at the next run when they exceed `minIdle`. A returned agent is destroyed instead of kept when the idle queue already holds `maxSize` agents.

`preWarm(count)` creates up to `count` idle agents without exceeding `maxSize` and returns how many it created. `close()` stops the evictor and destroys the agents.

### Metrics and listeners

`getMetrics()` returns a `PoolMetrics` record with `active`, `idle`, `totalCreated`, `totalDestroyed` and `peakActive`. `addListener` and `removeListener` manage `PoolListener` instances. Its methods `onActivated`, `onReturned`, `onDestroyed` and `onExhausted` all default to no-ops.

### Predictive pre-warming

`PredictiveActivator` registers itself as a listener on the pool. After `start()`, it checks the last `windowSize` seconds of checkouts every `analyzeInterval` seconds. When the window holds at least `burstThreshold` checkouts, or any exhaustion, it tops the idle count up to `preWarmCount` with `preWarm`. The defaults are a 60-second window, a threshold of 5, 3 agents and a 10-second interval. `stop()` ends the checks and removes the listener. `recentActivationCount()` and `recentExhaustionCount()` report the current window.

Because the evictor removes idle agents above `minIdle`, pre-warmed agents last only until the next eviction unless `minIdle` covers them.

## Related

- [Choosing a topology](choose-topology.md)
- [Council and voting](council.md)
- [Judge](judge.md)