# External LLM Configuration

An agent's provider, model and tuning normally come from annotations, which are compiled into the artifact. `LLMConfigurationSource` adds a layer above them that an operator can change without recompiling: one JSON file per agent, or a set of environment variables.

Register a source on the bridge:

```java
SCOPBridge bridge = SCOPBridge.getInstance();

// One JSON file per agent, re-read on every resolution
bridge.llmConfigurationSource(LLMConfigurationSource.folder(Path.of("/etc/tnsai/llm")));

// Or: TNSAI_LLM_<AGENT>_<FIELD> from System properties and the environment
bridge.llmConfigurationSource(LLMConfigurationSource.environment());

// Or: explicitly nothing, so the annotations stand alone
bridge.llmConfigurationSource(LLMConfigurationSource.noOp());
```

A folder source reads `<folder>/<agentName>.json`:

```json
{
  "provider": "OPENAI",
  "model": "gpt-4o",
  "temperature": 0.0,
  "maxTokens": 2048
}
```

The keys are `provider`, `model`, `temperature`, `maxTokens`, `endpoint` and `apiKeyEnv` — every one optional. A key that is not in that set is an error, not a no-op, so a misspelled `temprature` fails loudly instead of quietly doing nothing.

> The examples on this page are plain snippets, not compiled contract fixtures. These types are not in a published artifact yet, and the unreleased-main contract lane compiles against a pinned framework commit that predates them.

## Where a value can come from

Four layers contribute, lowest precedence first:

1. **Playground** — `@AgentSpec(llm = @LLMSpec(...))` on the playground class
2. **Agent** — `@AgentSpec(llm = @LLMSpec(...))` on the agent class
3. **Role** — `@RoleSpec(llm = @LLMSpec(...))` on the role class
4. **External source** — the registered `LLMConfigurationSource`

They do not all combine the same way. There are two rules, not one, and reading them as a single "higher layer overrides lower" rule is how you end up sending a prompt to the wrong provider.

### Routing is taken as a unit

`provider`, `model`, `endpoint` and `apiKeyEnv` are **routing**. They come as a set from the highest-precedence annotation tier that declares a `model`. No other tier contributes any part of the routing — not its provider, not its endpoint.

A tier that names a routing field but no model contributes nothing, and logs a warning naming what it dropped.

```java
@AgentSpec(llm = @LLMSpec(provider = Provider.OPENAI, model = "gpt-4o", apiKeyEnv = "OPENAI_API_KEY"))
public class SupportAgent extends Agent { }

@RoleSpec(llm = @LLMSpec(model = "llama3.2"))
public class SupportRole extends Role { }

// Role declares a model, so Role owns the whole routing set:
//   model    = llama3.2
//   provider = OLLAMA   <- @LLMSpec's own default, NOT the agent's OPENAI
//   endpoint = the Ollama default
```

The reason is worth stating plainly, because the alternative looks more helpful and is not: a model belongs to a provider, an endpoint speaks one provider's wire format, and a key is issued by one provider. Merging the four fields across tiers written by different authors produces a configuration nobody wrote. In the example above, a per-field merge would take the role's local model name and post it to `api.openai.com` under the agent's key.

If no layer supplies a model, resolution yields an empty result and warns. That is unchanged.

### Tuning merges field by field

`temperature` and `maxTokens` are **tuning**. They merge independently across every tier, with higher precedence winning per field. A tier that declares only a temperature now contributes it, which is what makes a role able to say "be deterministic" without restating the model.

The external source is the only layer that overlays *every* field individually, routing included. It is one operator configuring one deployment, so requiring them to restate the provider in order to change a temperature is the duplication this layer removes.

## `temperature = 0.0` is a value, not a blank

Zero is a legitimate temperature and zero is a legitimate `maxTokens`. A plain `float` cannot tell "not overridden" from "set to zero", so every field is carried as an `Optional`, `OptionalDouble` or `OptionalInt`:

```java
LLMOverrides deterministic = LLMOverrides.builder().temperature(0.0).build();
// temperature is present and 0.0 — it survives the merge and reaches the client
```

Write `"temperature": 0.0` in a source file and it applies.

## A member set to its own default reads as unwritten

`@LLMSpec`'s members have real defaults, not sentinels: `provider` defaults to `OLLAMA`, `temperature` to `0.7f`, `maxTokens` to `4096`. Only `model`, `endpoint` and `apiKeyEnv` default to the empty string. Every annotated class therefore carries a full set of values whether or not its author wrote any of them, and annotation retention keeps no record of which members were actually typed.

So a member counts as **declared** only when it differs from the default `@LLMSpec` itself declares. The consequence: writing `temperature = 0.7f` explicitly is indistinguishable from not writing it, and a lower tier's temperature will win.

This is harmless for routing, because routing comes whole from one tier and no competing value can reach the configuration. It is **not** harmless for tuning, which still merges. If you need a value that happens to equal the default, set it from a source rather than an annotation.

## When a source cannot be read

Nothing falls back. Answering with a model other than the configured one is worse than not answering, so `LLMConfigurationException` is thrown for:

- a file that cannot be read, or is not valid JSON
- a JSON value of the wrong type
- an unknown key
- a blank string value
- a file larger than 1 MiB
- a `maxTokens` that is fractional or outside `int` range
- a `temperature` outside 0.0–2.0
- a `provider` the bridge cannot route

A **missing** file is not an error. The source returns nothing and the annotations stand.

Validation applies to sources only. Annotation values are never re-validated: `@LLMSpec(maxTokens = -1)` is an established way to say "let the provider decide", and rejecting a value that is already compiled and shipped would break working deployments.

## Which agent a source is asked about

The lookup key is the agent's own `getName()`, falling back to the simple class name. Pass it explicitly through the four-argument `resolveLLMSpec` overload when you need a different one.

Two things to know about folder lookup:

- **Case sensitivity is the filesystem's, not ours.** `Researcher.json` and `researcher.json` are one entry on macOS and two on Linux. A configuration that works on a developer's Mac can stop applying in a Linux container.
- An agent name that cannot be a file name — a path separator, `..`, a NUL — resolves to nothing and logs. It is never resolved against the filesystem.

## Environment source keys

`LLMConfigurationSource.environment()` reads `TNSAI_LLM_<AGENT>_<FIELD>`, where `<AGENT>` is the agent name upper-cased with every non-alphanumeric character replaced by `_`, and `<FIELD>` is one of `PROVIDER`, `MODEL`, `TEMPERATURE`, `MAX_TOKENS`, `ENDPOINT`, `API_KEY_ENV`:

```bash
TNSAI_LLM_RESEARCHER_MODEL=gpt-4o
TNSAI_LLM_RESEARCHER_TEMPERATURE=0.0
```

Because the key is normalised, agent names differing only in punctuation — `data-analyst` and `data_analyst` — map to the same variable.

Values are read on every call for System properties and environment variables. They are **not** re-read from a `.env` file, which is loaded once per JVM: editing it, including removing an override, takes effect only after a restart.

## Trust boundary

**A configuration source is trusted infrastructure, not user input.** Give it the same trust as the deployment's own configuration.

`apiKeyEnv` names *any* environment variable or System property the process can see, and `endpoint` decides where that value is sent as a bearer token, along with the request body. Whoever can write the configuration folder can therefore read and exfiltrate a secret that has nothing to do with LLMs, without any access to the code.

Do not point a source at a directory that users, uploads or a web request can write to, and do not build a feature that lets an end user supply these fields.

A source can also supply a `model` for an agent whose annotations declare none, which means it can enable LLM dispatch for an agent the developer never configured for it.

Each resolution logs the lookup key, which fields the source contributed, and the `endpoint` and `apiKeyEnv` values when it sets them. None of those is itself a secret — `apiKeyEnv` is a variable name, not its value.

## Limitations

- **An endpoint pin is ignored on the Core SPI client path.** When `tnsai-llm` is on the classpath, client creation carries neither `endpoint` nor `apiKeyEnv`, so a configured endpoint is dropped and traffic goes to the provider default. A warning is logged. Tracked as TAN-6202.
- **`HUGGINGFACE` has no default endpoint of its own.** A source naming it is refused, because it would otherwise fall back to the Ollama endpoint carrying whatever `apiKeyEnv` came with it. Annotations are not checked, so `@LLMSpec(provider = Provider.HUGGINGFACE)` still misroutes. Pre-existing.
- **`maxTokens = 0` does not mean the same thing everywhere.** Through the Core SPI path it means "let the provider decide". On the HTTP fallback path for Anthropic it becomes a hardcoded `4096`.
- Provider matching is case-insensitive and normalised with `Locale.ROOT`, so `"openai"` resolves to `OPENAI`.

## Related

- [Providers](providers.md) — the wired clients and their default endpoints
- [Routing](routing.md) — choosing between models per request
- [SCOP Bridge](../../integrate/scop.md) — where resolution runs
- [Configuration Reference](../../reference/configuration.md) — environment variables