Supervision and safeguards
tnsai-coordination has two kinds of safeguards for agent groups. A SupervisorAgent answers "may this agent do this?" through a list of guardrails, and an AgentRecoveryManager watches agent health. Separate utilities isolate failing agents, cap token spend, keep an episode log and run a task in a sandbox.
Supervisor guardrails
SupervisorAgent in com.tnsai.coordination.supervisor wraps an AgentGroup and a list of Guardrails. It does not intercept agent calls: your code asks checkAction(agentId, action, context) before it lets an agent act, and acts on the answer.
This Guardrail is about action permission. It is not @InputGuardrail and not the content-evaluation Guardrail used by tracing; see Guardrails for the three kinds.
A guardrail implements getName() and check(agentId, action, context), and returns a GuardrailResult(allowed, reason, severity):
| Factory | allowed | severity |
|---|---|---|
GuardrailResult.allow() | true | INFO |
GuardrailResult.warning(reason) | true | WARNING |
GuardrailResult.blocked(reason) | false | BLOCK |
checkAction runs the guardrails in order:
- With no guardrails, it allows.
- The first result that is not allowed is returned at once, and later guardrails do not run.
- A guardrail that throws counts as a block with reason
Guardrail error: …. - A warning is logged and evaluation continues. If every guardrail allows, the result is the last guardrail's, so a warning from an earlier guardrail is not in the returned result.
addGuardrail(guardrail) appends one, removeGuardrail(name) removes every guardrail with that name, and getGuardrails() returns a copy of the list.
Human approval
HITLApproval is a guardrail that asks a person. Create it with a set of action patterns and a callback, and optionally a name (default hitl-approval). An action matches when it contains any pattern, ignoring case, so the pattern deploy also matches redeploy-cache. Non-matching actions are allowed.
For a matching action, the guardrail calls the callback with an ApprovalRequest(agentId, action, context, approval) and then blocks the calling thread until the approval future completes. Complete it with true to allow or false to block. There is no timeout, so make sure every request gets an answer. A callback exception or an exceptionally completed future blocks the action. addPattern and removePattern change the patterns at runtime.
package com.example.tnsai.docs;
import com.tnsai.agents.groups.AgentGroup;
import com.tnsai.coordination.supervisor.AgentRecoveryManager;
import com.tnsai.coordination.supervisor.AgentRecoveryManager.HealthStatus;
import com.tnsai.coordination.supervisor.AgentRecoveryManager.RecoveryAction;
import com.tnsai.coordination.supervisor.AgentRecoveryManager.RecoveryPolicy;
import com.tnsai.coordination.supervisor.AgentRecoveryManager.RecoveryResult;
import com.tnsai.coordination.supervisor.Guardrail;
import com.tnsai.coordination.supervisor.Guardrail.GuardrailResult;
import com.tnsai.coordination.supervisor.HITLApproval;
import com.tnsai.coordination.supervisor.SupervisorAgent;
import java.time.Duration;
import java.util.List;
import java.util.Map;
import java.util.Set;
/** Compilation contract for multi-agent/supervision.md supervisor and recovery. */
public final class SupervisionExample {
private SupervisionExample() {
}
/** Blocks any action whose context carries a cost above a fixed cap. */
static final class CostCap implements Guardrail {
@Override
public String getName() {
return "cost-cap";
}
@Override
public GuardrailResult check(String agentId, String action, Map<String, Object> context) {
Object cost = context.get("cost");
if (cost instanceof Number n && n.doubleValue() > 100.0) {
return GuardrailResult.blocked("Cost " + n + " is over the cap");
}
return GuardrailResult.allow();
}
}
static boolean mayDeploy(AgentGroup group) {
HITLApproval approval = new HITLApproval(Set.of("deploy", "delete"), request -> {
// Ask a person here; complete the future with their answer.
System.out.println(request.agentId() + " wants to " + request.action());
request.approval().complete(Boolean.TRUE);
});
SupervisorAgent supervisor = new SupervisorAgent(group, List.of(new CostCap(), approval));
GuardrailResult result = supervisor.checkAction(
"release-agent", "deploy", Map.of("target", "production", "cost", 40));
if (!result.allowed()) {
System.out.println("Blocked: " + result.reason() + " (" + result.severity() + ")");
}
return result.allowed();
}
static AgentRecoveryManager monitor(SupervisorAgent supervisor) {
AgentRecoveryManager recovery = new AgentRecoveryManager(supervisor, RecoveryPolicy.builder()
.maxRetries(3)
.escalateAfter(2)
.cooldown(Duration.ofSeconds(10))
.defaultAction(RecoveryAction.REPLACE)
.build());
recovery.addHealthCheck("worker-1", agent -> !agent.chat("ping").isBlank());
recovery.addListener(new AgentRecoveryManager.RecoveryListener() {
@Override
public void onHealthCheckFailed(HealthStatus status) {
System.out.println("Unhealthy: " + status.agentId() + " - " + status.reason());
}
@Override
public void onRecoveryAttempt(RecoveryResult result) {
System.out.println(result.actionTaken() + ": " + result.detail());
}
@Override
public void onEscalation(String agentId, int failedAttempts) {
System.out.println("Escalate " + agentId + " after " + failedAttempts + " attempts");
}
});
recovery.start(Duration.ofSeconds(30)); // call stop() when done
return recovery;
}
}Health checks and recovery
AgentRecoveryManager takes the supervisor and a RecoveryPolicy. Register a HealthCheck per agent id with addHealthCheck(agentId, check); a check is a predicate on the Agent. start(interval) runs every check on a background thread at a fixed rate until stop(). runHealthChecks() runs them once and returns a HealthStatus per agent. Agents that are no longer members of the group are skipped.
A failed or throwing check, or a call to triggerRecovery(agentId, reason), starts a recovery attempt:
- Within the cooldown of the agent's previous attempt, nothing happens and a failed
RecoveryResultis returned. - Otherwise the action is chosen from the agent's retry count:
EJECTonce it reachesmaxRetries,ESCALATEonce it reachesescalateAfter, and the policy'sdefaultActionbefore that. - The retry count goes up by one, and the result is added to the history and sent to listeners.
RecoveryPolicy builder | Default |
|---|---|
maxRetries | 3 |
escalateAfter | 3 |
cooldown | 5 seconds |
defaultAction | RESTART |
With the defaults, maxRetries and escalateAfter are equal, so an agent is ejected without being escalated. Set escalateAfter below maxRetries to escalate first.
RecoveryAction | Effect |
|---|---|
EJECT | Removes the agent from the group, with its health check and retry count |
REPLACE | The same removal; adding a replacement is up to you |
RESTART, RESTART_CLEAN | Reported as successful but change nothing; react to them in a listener |
ESCALATE | Changes nothing and calls onEscalation on the listeners |
A RecoveryListener has onHealthCheckFailed, onRecoveryAttempt and onEscalation, and all three must be implemented. The retry count is not reset when an agent becomes healthy again; call resetRetryCounter(agentId) for that. getRecoveryHistory() and getRetryCount(agentId) report what happened.
Production safeguards
The utilities in com.tnsai.coordination.production, and SandboxedExecution, are independent of the supervisor. Use them around your own agent calls.
Failure isolation
FailureIsolator keeps a circuit breaker per agent id. execute(agentId, action, fallback) runs the action:
- A success resets the failure count and closes the circuit.
- A failure below
failureThresholdis rethrown. - The failure that reaches the threshold opens the circuit and returns the degraded value instead of throwing.
- While the circuit is
OPEN, calls return the degraded value without running the action. - After
recoveryTimeoutthe circuit isHALF_OPENand calls run the action again. A success closes it; a failure opens it again.
DegradationStrategy | Degraded value |
|---|---|
FALLBACK (default) | The fallback supplier's value, or null without one |
SKIP | null |
CACHED | The agent's last successful result, or null |
FAIL_FAST | Throws FailureIsolator.CircuitOpenException |
Builder defaults are a threshold of 3 failures and a 30-second recovery timeout. state(agentId), failureCount(agentId) and reset(agentId) inspect and reset a circuit.
Token budgets
TokenBudget tracks token use per agent against an allocation and a global budget. It records and reports; it does not stop anything.
allocate(agentId, tokens)sets an agent's allocation. Allocating again replaces it and starts the agent's count from zero.consume(agentId, tokens)adds to the agent's and the global count. It throwsIllegalStateExceptionfor an agent with no allocation, and it records usage beyond the budget, logging a warning from 90% usage.canConsume(agentId, tokens)is true when both the agent's allocation and the global budget have room. Check it before each call.remaining,globalRemaining,usagePercent,globalUsagePercentandallAllocationsreport the state.
Episodic memory
EpisodicMemory(maxEpisodes) is a bounded log of Episode(taskId, agentId, taskType, outcome, metadata, timestamp) records. Episode.of(...) stamps the current time. When the log is full, record drops the oldest episode. The queries return the newest episodes first, up to a limit:
querySimilar(taskType, limit): the task type contains the text, ignoring case;queryByAgent(agentId, limit);queryByOutcome(outcome, limit): the outcome equals the text, ignoring case.
Sandboxed execution
SandboxedExecution.execute(agent, task, config) runs agent.chat(task) on its own thread and waits up to the config's timeout. It returns a SandboxResult(result, completed, duration, error). A timeout, interruption or exception gives completed = false with an error message; it does not throw.
SandboxConfig(timeout, maxMemoryBytes, allowedActions, readOnly) has defaults() of 30 seconds, 0, no actions and not read-only. Only timeout and readOnly have an effect. readOnly prefixes the task with [READ-ONLY MODE]; it is an instruction to the model, not an enforced restriction. To run untrusted code in isolation, see Sandbox.
package com.example.tnsai.docs;
import com.tnsai.agents.Agent;
import com.tnsai.coordination.production.EpisodicMemory;
import com.tnsai.coordination.production.EpisodicMemory.Episode;
import com.tnsai.coordination.production.FailureIsolator;
import com.tnsai.coordination.production.FailureIsolator.DegradationStrategy;
import com.tnsai.coordination.production.TokenBudget;
import com.tnsai.coordination.supervisor.SandboxedExecution;
import com.tnsai.coordination.supervisor.SandboxedExecution.SandboxConfig;
import com.tnsai.coordination.supervisor.SandboxedExecution.SandboxResult;
import java.time.Duration;
import java.util.List;
import java.util.Map;
import java.util.Set;
/** Compilation contract for multi-agent/supervision.md production safeguards. */
public final class ProductionHardeningExample {
private ProductionHardeningExample() {
}
static String answer(Agent agent, String question) {
FailureIsolator isolator = FailureIsolator.builder()
.failureThreshold(3)
.recoveryTimeout(Duration.ofSeconds(30))
.degradationStrategy(DegradationStrategy.FALLBACK)
.build();
TokenBudget budget = new TokenBudget(100_000);
budget.allocate(agent.getId(), 30_000);
EpisodicMemory memory = new EpisodicMemory(1_000);
if (!budget.canConsume(agent.getId(), 2_000)) {
return "Budget exhausted";
}
String reply = isolator.execute(agent.getId(),
() -> agent.chat(question),
() -> "The assistant is unavailable, try again later.");
budget.consume(agent.getId(), 2_000); // record what the call used
memory.record(Episode.of("q-1", agent.getId(), "faq", "success", Map.of("chars", "" + reply.length())));
List<Episode> recent = memory.queryByAgent(agent.getId(), 10);
System.out.println(recent.size() + " episodes, circuit " + isolator.state(agent.getId())
+ ", " + budget.remaining(agent.getId()) + " tokens left");
return reply;
}
static SandboxResult dryRun(Agent agent, String task) {
SandboxResult result = SandboxedExecution.execute(agent, task,
new SandboxConfig(Duration.ofSeconds(20), 0, Set.of(), true));
if (!result.completed()) {
System.out.println("Failed after " + result.duration() + ": " + result.error());
}
return result;
}
}