# Cost

TnsAI has two cost layers:

| | Client cost tracking | Tenant budgets |
|---|---|---|
| Module and package | `tnsai-llm`, `com.tnsai.llm.cost` | `tnsai-quality`, `com.tnsai.quality.cost` |
| Main types | `CostAwareLLMClient`, `CostTracker`, `BudgetManager` | `CostBudget`, `CostBudgetStore`, `CostBudgetEnforcer` |
| Scope | One wrapped client | A tenant, an agent in a tenant, or a capability |
| Enforcement | `BudgetExceededException` before a call | A `Block` from a hook you dispatch |

## Track a client's cost

Wrap a client with `CostAwareLLMClient`. Each call is recorded as a `UsageRecord`, priced from `ModelPricing`.

```java
import com.tnsai.llm.cost.CostAwareLLMClient;
import com.tnsai.llm.cost.InMemoryCostTracker;

CostAwareLLMClient tracked = CostAwareLLMClient.wrap(client);

// Or with every option:
CostAwareLLMClient tracked = CostAwareLLMClient.builder()
    .client(client)
    .costTracker(InMemoryCostTracker.builder().maxRecords(10_000).build())
    .budgetManager(budget)
    .build();
```

| Builder | Default | Meaning |
|---|---|---|
| `client(LLMClient)` | Required | `build()` throws `IllegalStateException` without it |
| `costTracker(CostTracker)` | `InMemoryCostTracker.create()` | Where records go |
| `budgetManager(BudgetManager)` | None | Spend limit, see below |
| `estimateBeforeRequest(boolean)` | `true` | Check the budget before a call |
| `estimatedInputTokensPerChar(int)` | `4` | Characters per token, for estimates |

`wrap(client, budget)` is the short form with a budget. `InMemoryCostTracker.maxRecords` defaults to 0, which keeps every record. Over the limit, the oldest record is dropped.

### What gets recorded

- **Model:** the record uses the wrapped client's `getModel()`.
- **Tokens:** `chat` takes token counts from the response. When the response has none, both input and output are estimated from the response length, divided by `estimatedInputTokensPerChar`.
- **Streaming:** each streamed chunk counts as one output token, and input is estimated from the prompt length. The record is written when you close the stream.
- **Price:** `ModelPricing.forModel` matches the model ID exactly, else by the longest registered prefix. `gpt-4o-2024-08-06` is priced as `gpt-4o`. A model with no match costs 0.

### Read the totals

From the client:

| Method | Returns |
|---|---|
| `getTotalCost()` | `BigDecimal`, USD |
| `getRequestCount()`, `getTotalInputTokens()`, `getTotalOutputTokens()` | `long` |
| `getStatistics()` | `CostTracker.CostStatistics` |
| `getBudgetStatus()` | `Optional<BudgetManager.BudgetStatus>` |
| `getCostTracker()`, `getBudgetManager()`, `getDelegate()` | The parts. `getBudgetManager()` is an `Optional`. |

From the tracker:

```java
import com.tnsai.llm.cost.CostTracker;
import java.math.BigDecimal;
import java.time.Instant;
import java.time.temporal.ChronoUnit;

CostTracker tracker = tracked.getCostTracker();

BigDecimal total = tracker.getTotalCost();
BigDecimal gpt4o = tracker.getCostByModel().get("gpt-4o");
BigDecimal lastHour = tracker.getTotalCost(
    Instant.now().minus(1, ChronoUnit.HOURS), Instant.now());
```

| `CostTracker` method | Returns |
|---|---|
| `record(UsageRecord)` | |
| `getRecords()`, `getRecords(from, to)` | `List<UsageRecord>` |
| `getRecordsByModel(modelId)`, `getRecordsByProvider(provider)` | `List<UsageRecord>` |
| `getTotalCost()`, `getTotalCost(from, to)` | `BigDecimal` |
| `getStatistics()`, `getStatistics(from, to)` | `CostStatistics` |
| `getCostByModel()`, `getCostByProvider()` | `Map<String, BigDecimal>` |
| `getRecordCount()`, `clear()` | |

Time ranges include `from` and exclude `to`.

`CostStatistics` holds the record count, token totals (input, output, cached), input, output and total cost, the average cost and latency per request, the most expensive request, cost by model and by provider, and the time span.

### Record usage yourself

Record calls made outside a wrapped client:

```java
import com.tnsai.llm.cost.UsageRecord;

tracker.record(UsageRecord.builder()
    .modelId("gpt-4o")
    .inputTokens(1000)
    .outputTokens(500)
    .build());
```

`modelId` is required. `build()` prices the record, and `getProvider()` comes from the pricing entry unless you set `provider`. The builder also takes `cachedInputTokens`, `latencyMs`, `timestamp`, `id` and `metadata`. Read the cost with `getInputCost()`, `getOutputCost()`, `getCachedInputCost()` and `getTotalCost()`.

## Limit a client's spend: `BudgetManager`

```java
import com.tnsai.llm.cost.BudgetManager;

BudgetManager budget = BudgetManager.builder()
    .limit("50.00")
    .daily()
    .defaultAlertThresholds()
    .onAlert(alert -> System.out.println(alert))
    .build();
```

| Builder | Default | Meaning |
|---|---|---|
| `limit(BigDecimal \| String \| double)` | None, unlimited | USD per period |
| `daily()`, `monthly()`, `period(Duration)` | None, never resets | `monthly()` is 30 days |
| `alertThreshold(double)`, `alertThresholds(double...)` | None | Fractions of the limit, such as `0.8` |
| `defaultAlertThresholds()` | | 0.5, 0.8, 0.9 and 0.95 |
| `hardLimit(boolean)`, `softLimit()` | Hard | A soft limit never blocks |
| `onAlert(Consumer<BudgetAlert>)` | None | Called once per threshold per period, and on every spend that takes the total over the limit |
| `onLimitExceeded(Consumer<BigDecimal>)` | None | Called with the new total on every spend over the limit |
| `costTracker(CostTracker)` | None | Load the current period's spend from a tracker at start and on each period reset. Needs a period. |

### When a call is blocked

Before a `chat(String, …)` call, `CostAwareLLMClient` estimates the cost: the prompt, system prompt and history length for input, and the client's max tokens (else 1000) for output. With a hard limit, it throws `com.tnsai.llm.cost.BudgetExceededException` when that estimate would take spend over the limit. The exception carries `getCurrentSpend()`, `getLimit()`, `getEstimatedCost()` and `getRemaining()`.

Only that `chat` overload checks. Multimodal `chat`, `chat` with a temperature, and the streaming methods record spend but never block. With `estimateBeforeRequest(false)`, nothing blocks. A model with no price estimates to 0, so it is blocked only once recorded spend is already over the limit.

### Status

`getStatus()` returns a `BudgetStatus(state, currentSpend, limit, remaining, usagePercent, periodStart, remainingTime)`.

| `BudgetState` | Usage |
|---|---|
| `OK` | Below 70% |
| `WARNING` | 70% to 90% |
| `CRITICAL` | 90% to 100% |
| `EXCEEDED` | 100% or more |
| `UNLIMITED` | No limit set |

The period restarts on the first budget call after it ends, which clears the spend and the triggered alerts. `reset()` restarts it at once. `canSpend(amount)` checks without recording. `trySpend(amount)` records only when the amount fits.

## Model prices

`ModelPricing` holds about 50 models, in USD per million tokens. A sample:

| Model ID | Input | Output | Cached input |
|---|---|---|---|
| `gpt-4o` | 2.50 | 10.00 | 1.25 |
| `gpt-4o-mini` | 0.15 | 0.60 | 0.075 |
| `gpt-4.1` | 2.00 | 8.00 | 0.50 |
| `claude-sonnet-4-20250514` | 3.00 | 15.00 | 0.30 |
| `claude-opus-4-20250514` | 15.00 | 75.00 | 1.50 |
| `claude-haiku-4-5-20251001` | 0.80 | 4.00 | 0.08 |
| `gemini-2.5-pro` | 1.25 | 10.00 | 0.3125 |
| `gemini-2.5-flash` | 0.15 | 0.60 | 0.0375 |
| `llama-3.3-70b-versatile` | 0.59 | 0.79 | |
| `llama3.2`, `qwen2.5`, `mistral` (Ollama) | 0 | 0 | |

The table also covers older OpenAI, Anthropic and Gemini models, Groq, Mistral, OpenRouter, ZhipuAI and MiniMax. A model ID is priced only if it starts with a registered ID, so `claude-sonnet-4` alone has no price.

```java
import com.tnsai.llm.cost.ModelPricing;
import java.math.BigDecimal;
import java.util.Optional;

Optional<ModelPricing> pricing = ModelPricing.forModel("gpt-4o");
BigDecimal cost = pricing
    .map(p -> p.calculateTotalCost(1000, 500))
    .orElse(BigDecimal.ZERO);

// Add or replace a price, per million tokens.
ModelPricing.registerPricing("my-model", "MyProvider",
    new BigDecimal("1.00"), new BigDecimal("2.00"));
```

`ModelPricing` also has `calculateInputCost`, `calculateOutputCost`, `calculateCachedInputCost`, `calculateTotalCost(input, cached, output)`, `getInputPricePerMillion`, `getOutputPricePerMillion`, `getCachedInputPricePerMillion` and `supportsCachedPricing`.

## Tenant budgets

Tenant budgets keep spend in a shared store, so one cap can cover many clients and agents. You record spend into the store and check it before each call.

### Budgets and scopes

```java
import com.tnsai.quality.cost.BudgetPolicy;
import com.tnsai.quality.cost.CostBudget;
import java.math.BigDecimal;
import java.time.Duration;

CostBudget tenantCap = CostBudget.tenantWide("acme",
    new BigDecimal("500.00"), Duration.ofDays(1), BudgetPolicy.HARD_STOP);

CostBudget summarizerCap = CostBudget.forAgent("acme", "summarizer-agent",
    new BigDecimal("50.00"), Duration.ofDays(1), BudgetPolicy.SOFT_WARN);
```

`CostBudget(tenantId, agentId, capabilityId, periodBudgetUSD, period, onExceed)` takes an `Optional` agent and capability. The budget and period must be positive, and the tenant must not be blank.

Spend is recorded against a `CostScope`: `CostScope.tenant(t)`, `CostScope.agent(t, a)` or `CostScope.full(t, a, c)`. A budget sees every spend inside its scope. A tenant budget counts all of the tenant's agents, and an agent budget counts all of that agent's capabilities.

### Record and evaluate

```java
import com.tnsai.quality.cost.BudgetState;
import com.tnsai.quality.cost.CostBudgetEvaluator;
import com.tnsai.quality.cost.CostBudgetStore;
import com.tnsai.quality.cost.CostScope;
import com.tnsai.quality.cost.InMemoryCostBudgetStore;

CostBudgetStore store = new InMemoryCostBudgetStore(Duration.ofDays(31));
CostBudgetEvaluator evaluator = new CostBudgetEvaluator(store);

// After each call, for example from an LLMCallLog named call:
store.record(CostScope.full("acme", "summarizer-agent", "extractive-summary"),
    call.cost().totalUSD());

// Before the next call:
BudgetState state = evaluator.evaluate(tenantCap);
switch (state.status()) {
    case HEALTHY -> { }
    case WARNING -> System.out.println("At " + (int) (state.fractionConsumed() * 100) + "%");
    case EXHAUSTED -> throw new IllegalStateException("acme is over its daily cap");
}
```

`InMemoryCostBudgetStore` keeps entries for `maxAge`, so set it at least as long as your longest budget period. A budget period is a rolling window: the last 24 hours for `Duration.ofDays(1)`, not a calendar day.

`BudgetState` holds `budget`, `spentUSD`, `remainingUSD`, `fractionConsumed`, `status` and `evaluatedAt`. The status is `HEALTHY` below 80%, `WARNING` from 80% to 100%, and `EXHAUSTED` at 100% or more.

To share spend across processes, implement `CostBudgetStore`: `record(scope, amountUSD)`, `currentSpend(scope, period)` and `clear()`.

### Enforce with a hook

`CostBudgetEnforcer` is a `Hook<PreAction>` that checks a budget before LLM actions:

```java
import com.tnsai.hooks.HookDispatcher;
import com.tnsai.hooks.HookFailurePolicy;
import com.tnsai.quality.cost.CostBudgetEnforcer;
import java.util.Map;

Map<String, CostBudget> budgetByTenant = Map.of("acme", tenantCap);

CostBudgetEnforcer enforcer = new CostBudgetEnforcer(store,
    event -> event.context().tenantId().map(budgetByTenant::get));

HookDispatcher dispatcher = new HookDispatcher(Duration.ofSeconds(5), HookFailurePolicy.FAIL_CLOSED);
dispatcher.register(enforcer);
```

- **Matching:** it handles only `PreAction` events of type `ActionType.LLM`, at priority -100, before other hooks.
- **Resolver:** the second argument maps an event to its budget. With no budget, the call is allowed.
- **Estimate:** an optional third argument, a `Function<PreAction, BigDecimal>`, estimates the call's cost. The two-argument form uses 0, so a call is caught only once recorded spend reaches the cap.
- **Trigger:** the policy applies when spend plus the estimate reaches the budget.

| Policy | Result |
|---|---|
| `SOFT_WARN` | Logs a warning and allows the call |
| `HARD_STOP` | `HookResult.Block` with severity `HIGH` and a budget-exceeded reason |
| `DEFER` | Blocks like `HARD_STOP`. The reason says the caller must reschedule. TnsAI has no queue. |

TnsAI does not dispatch `PreAction` events itself. Call `dispatcher.dispatch(event)` before each LLM call and stop on a `Block`. The default `HookDispatcher()` is fail-open: a hook that throws or times out allows the call. Use `FAIL_CLOSED` for enforcement.

## Related

- [LLM calls](../observability/llm-calls.md): `LLMCallLog` and its cost estimate.
- [Caching](caching.md): cut cost with response and prompt caching.
- [Routing](routing.md): pick a client by cost.