A router holds several clients and picks one per request. Every router implements LLMRouter, which extends LLMClient, so you pass a router anywhere a client goes, including AgentBuilder.llm(...). The routers live in tnsai-llm, in the package com.tnsai.llm.routing.

Router	Picks	Moves to the next client on failure
`FallbackRouter`	The clients in the order you list them	Yes
`CostBasedRouter`	The lowest declared input price first	Yes
`LatencyBasedRouter`	The lowest average latency first	Yes
`CapabilityRouter`	The clients that meet a capability requirement	Yes
`RoundRobinRouter`	The next client in rotation	Yes
`TaskBasedRouter`	The client mapped to the message's task type	No

Each router needs at least one client, or it throws IllegalArgumentException. When every client fails, the router throws LLMException with the last failure as its cause.

Prices and latencies come from the model table

CostBasedRouter, LatencyBasedRouter and CapabilityRouter read client.getCapabilities(). The provider clients answer from a built-in table of known models, matched on the model name, so a model the table does not know gets conservative defaults:

no price, so CostBasedRouter tries it last;
an estimated latency of 1000 ms until it has been measured;
no vision, tool calling or structured output, and a 4096-token input window.

Check what a client declares before you rely on a router:

LLMCapabilities caps = client.getCapabilities();
caps.getInputCostPer1KTokens();   // Optional<Double>
caps.getEstimatedLatencyMs();     // Optional<Long>
caps.supportsVision();

See Providers: capabilities for the full interface.

`FallbackRouter`

Tries the clients in order and moves on when one throws.

import com.tnsai.llm.providers.AnthropicClient;
import com.tnsai.llm.providers.GroqClient;
import com.tnsai.llm.providers.OpenAIClient;
import com.tnsai.llm.routing.FallbackRouter;
import com.tnsai.llm.routing.LLMRouter;

LLMRouter router = FallbackRouter.of(
    new OpenAIClient("gpt-4o"),
    new AnthropicClient("claude-sonnet-4-20250514"),
    new GroqClient("llama-3.3-70b-versatile"));

ChatResponse response = router.chat("Hello");
// OpenAI first, then Anthropic, then Groq.

of(...) takes clients or a List<LLMClient>. The builder adds a filter:

FallbackRouter router = FallbackRouter.builder()
    .addClient(new OpenAIClient("gpt-4o"))
    .addClient(new AnthropicClient("claude-sonnet-4-20250514"))
    .requireFunctionCalling()   // or requireVision(), or requireCapability(predicate)
    .build();

Clients that fail the filter are skipped. If none passes, the router has nothing to try and throws LLMException.

`CostBasedRouter`

Sorts the clients by declared input price per 1K tokens, cheapest first, and falls back to the next one on failure. Clients without a price go last.

import com.tnsai.llm.routing.CostBasedRouter;

CostBasedRouter router = CostBasedRouter.of(
    new OpenAIClient("gpt-4o"),                      // 0.0025 per 1K input tokens
    new OpenAIClient("gpt-4o-mini"),                 // 0.00015
    new GroqClient("llama-3.3-70b-versatile"));      // 0.00059
// Tries gpt-4o-mini, then Groq, then gpt-4o.

The builder takes addClient, requireCapability, requireVision and requireFunctionCalling. As with FallbackRouter, a filter that matches no client leaves nothing to try.

getCostStats() returns a CostStats record:

Component	Meaning
`totalEstimatedCost`	Estimated spend across all clients
`costPerProvider`	Estimated spend per `provider/model`
`totalInputTokens`	Estimated input tokens
`totalOutputTokens`	Estimated output tokens

The token counts are estimates at four characters per token, not the provider's usage figures. For billed usage, see Cost.

`LatencyBasedRouter`

Measures each call and sends the next request to the client with the lowest average over its last 20 calls. Before a client has been measured, its estimated latency from the model table stands in. A failed call is recorded as its duration plus 5000 ms, which pushes the client down the order.

import com.tnsai.llm.routing.LatencyBasedRouter;

LatencyBasedRouter router = LatencyBasedRouter.builder()
    .addClient(new GroqClient("llama-3.3-70b-versatile"))
    .addClient(new OpenAIClient("gpt-4o"))
    .maxLatencyMs(500)
    .build();

maxLatencyMs leaves out clients whose average is above the limit. The builder also takes requireCapability, requireVision and requireFunctionCalling. If the filters leave no client, the router uses all of them.

getLatencyStats() returns avgLatencyPerProvider, fastestProvider and fastestLatencyMs.

`CapabilityRouter`

Keeps the clients that meet a requirement you set on the builder, in the order you added them, and falls back down that list on failure. It does not look at the request: a message with an image does not select a vision model by itself. Set the requirement for the workload, or pick a client explicitly.

import com.tnsai.llm.providers.OllamaClient;
import com.tnsai.llm.routing.CapabilityRouter;

CapabilityRouter router = CapabilityRouter.builder()
    .addClient(new OllamaClient("llama3.2"))
    .addClient(new OpenAIClient("gpt-4o"))
    .requireVision()   // or requireFunctionCalling(), requireStructuredOutput(), defaultRequirement(predicate)
    .build();
// Only gpt-4o declares vision, so every request goes there.

If no client meets the requirement, the router uses all of them.

To pick a client yourself:

Method	Returns the first client that
`selectVisionCapable()`	Supports vision
`selectToolCapable()`	Supports function calling
`selectStructuredOutputCapable()`	Supports structured output
`selectWithMinContext(int minTokens)`	Has at least `minTokens` of input window
`selectByCapability(predicate)`	Matches the predicate

Each returns Optional<LLMClient>. selectByCapability is on every router.

`RoundRobinRouter`

Starts each request at the next client in rotation. On failure it tries the following clients, once each.

import com.tnsai.llm.providers.GeminiClient;
import com.tnsai.llm.routing.RoundRobinRouter;

RoundRobinRouter router = RoundRobinRouter.of(
    new OpenAIClient("gpt-4o"),
    new AnthropicClient("claude-sonnet-4-20250514"),
    new GeminiClient("gemini-2.5-flash"));

of(...) takes clients or a list. The builder takes addClient.

`TaskBasedRouter`

Classifies each message into a TaskType and sends it to the client mapped to that type. It sends one request and does not fall back: if that client throws, the exception reaches you. Wrap the mapped clients in a FallbackRouter when you need failover per task.

import com.tnsai.llm.routing.TaskBasedRouter;
import com.tnsai.llm.routing.TaskType;

TaskBasedRouter router = TaskBasedRouter.builder()
    .forTask(TaskType.CODING, new AnthropicClient("claude-sonnet-4-20250514"))
    .forTask(TaskType.MATH, new OpenAIClient("o1"))
    .forTask(TaskType.FAST, new GroqClient("llama-3.3-70b-versatile"))
    .defaultClient(new OpenAIClient("gpt-4o-mini"))
    .build();

router.chat("Write a Python function to sort a list");  // CODING, Anthropic
router.chat("Solve: integral of x squared dx");          // MATH, o1
router.chat("What is 2+2?");                             // MATH: "2+2" matches the equation pattern
router.chat("Write a poem about autumn");                // CREATIVE at 0.2, below 0.3, so the default client

Builder

Method	Meaning	Default
`forTask(type, client)`	Maps a task type to a client	None
`defaultClient(client)`	The client for unmapped types and low confidence. `build()` throws `IllegalStateException` without it.	Required
`classifier(classifier)`	A custom `TaskClassifier`	`TaskClassifier.defaultClassifier()`
`confidenceThreshold(threshold)`	Below this confidence, the message is routed as `GENERAL`	`0.3`
`withCommonSetup(coding, reasoning, fast, general)`	Maps `CODING`; `MATH` and `REASONING`; `FAST` and `CHAT`; and sets the default	None

GENERAL goes to the default client unless you map it with forTask.

Other methods

// Skip classification.
ChatResponse response = router.chatAs(TaskType.CODING, "Explain this concept");

// The client a type maps to, or the default client.
LLMClient coding = router.getClientForTask(TaskType.CODING);

// Classify without sending.
TaskClassifier.ClassificationResult result = router.classifyTask("Write a poem");

chat passes the content of each history message to the classifier. streamChat classifies the same way.

Statistics

getTaskStats() returns a TaskRoutingStats record:

Member	Meaning
`requestsPerTask()`	Requests per task type
`tokensPerTask()`	Estimated tokens per task type, at four characters per token
`estimatedSavings()`	Estimated savings against sending every token at 0.003 per 1K
`mostCommonTask()`	`Optional<TaskType>`
`taskDistribution()`	Percentage per task type
`routingStats()`	The `RoutingStats` below

resetStats() clears the task counters and the routing counters.

Task types

`TaskType`	Example keywords
`CODING`	code, function, class, debug, refactor, python, java, algorithm, api
`CREATIVE`	write a story, poem, fiction, marketing, slogan, blog post
`MATH`	calculate, solve, equation, integral, derivative, theorem
`ANALYSIS`	analyze, summarize, extract, compare, report, metrics
`TRANSLATION`	translate, convert to, in english, in french
`CHAT`	hello, hi, what is, explain, tell me, why
`FAST`	quick, simple, brief, yes or no, one word
`REASONING`	think step by step, reason, logic, deduce, infer
`VISION`	image, picture, photo, screenshot, diagram
`GENERAL`	None. The fallback type.

Each type also has Turkish keywords, such as kod, şiir and çevir. getKeywords() returns the full set.

Keywords match as lowercase substrings, so a short keyword can match inside a longer word: hi matches "this", and go matches "algorithm". matches(text) reports whether any keyword occurs, and matchScore(text) counts them. TaskType.fromId("coding") returns CODING, and an unknown id returns GENERAL.

`TaskClassifier`

TaskClassifier.defaultClassifier() scores every task type and picks the highest:

Signal	Points
A fenced code block	`CODING` +8
A declaration keyword, such as `def` , `function` , `public` or `import`	`CODING` +6
A source file extension, such as `.java` or `.py`	`CODING` +4
An arithmetic expression or equation, such as `2+2` or `x^2`	`MATH` +6
A math function name, such as `integral`, `sqrt` or `sin`	`MATH` +8
A reference to an image, such as "this image" or "the photo"	`VISION` +10
Each custom pattern that matches	+5
Each built-in keyword that occurs	+2
Each custom keyword that occurs	+3
A history message, of the last three, that contains a keyword of the type	+1.5

Confidence is the best score divided by 10, capped at 1.0. Below the classifier's own minimum, 0.1 by default, the result is the default type.

import com.tnsai.llm.routing.TaskClassifier;
import com.tnsai.llm.routing.TaskClassifier.ClassificationResult;

TaskClassifier classifier = TaskClassifier.defaultClassifier();

TaskType type = classifier.classify("Write a Python function to sort a list");   // CODING

ClassificationResult result = classifier.classifyWithConfidence("Solve x^2 + 2x + 1 = 0");
result.taskType();     // MATH
result.confidence();   // 0.8: equation 6, "solve" 2
result.reason();       // "Pattern and keyword match"
result.isConfident(0.5);

List<String> history = List.of("I'm working on a Java service", "Help me debug it");
TaskType withHistory = classifier.classify("What's wrong here?", history);

classify and classifyWithConfidence take an optional history list. An empty message returns the default type with confidence 0.

A custom classifier:

TaskClassifier classifier = TaskClassifier.builder()
    .addKeywords(TaskType.CODING, Set.of("backend", "frontend", "docker"))
    .addPattern(TaskType.MATH, Pattern.compile("\\d+%"))
    .setDefaultType(TaskType.CHAT)
    .setMinConfidenceThreshold(0.2)
    .build();

`RoutingStats`

Every router returns getStats(), and resetStats() clears it.

Component	Meaning
`totalRequests`	Requests received
`successfulRequests`	Requests that returned a response
`failedRequests`	Requests where every client failed
`requestsPerProvider`	Attempts per `provider/model`
`failuresPerProvider`	Failed attempts per `provider/model`
`avgLatencyPerProvider`	Average latency per `provider/model`, filled by the fallback and latency routers

Providers: the clients a router holds, and the circuit breaker.
Cost: billed usage and budgets.
LLM calls: observe each call.