# Images and speech

Three built-in toolkits in `tnsai-tools` let an agent generate images, speak text and transcribe audio. Each toolkit exposes one tool per provider, so the model can pick a provider at call time.

| Toolkit | `BuiltInTool` | Tools | Direction |
|---|---|---|---|
| `ImageGenTools` | `IMAGE_GEN_TOOLS` | `dalle3_generate`, `flux_generate`, `stability_generate` | Text to image |
| `TextToSpeechTools` | `TEXT_TO_SPEECH_TOOLS` | `elevenlabs_tts`, `cartesia_tts`, `deepgram_tts` | Text to speech |
| `SpeechToTextTools` | `SPEECH_TO_TEXT_TOOLS` | `deepgram_transcribe`, `assemblyai_transcribe`, `replicate_whisper` | Audio file to text |

`MEDIA_TOOLS` adds the OpenAI pair: `openai_tts` and `whisper_transcribe`. Register it next to the others when you want OpenAI and another provider in the same agent.

## Register the toolkits

Pass the toolkits to `builtInTools(...)`. `AgentBuilder.build()` requires `principal` and `authorityScope`, and `liabilitySink` is optional.

<!-- java-contract: src/main/java/com/example/tnsai/docs/ToolsMultimodalBuilderExample.java -->
```java
package com.example.tnsai.docs;

import com.tnsai.accountability.AuthorityScope;
import com.tnsai.accountability.LiabilitySink;
import com.tnsai.agents.Agent;
import com.tnsai.agents.AgentBuilder;
import com.tnsai.enums.BuiltInTool;
import com.tnsai.identity.AgentPrincipal;
import com.tnsai.llm.providers.OpenAIClient;
import com.tnsai.roles.Role;

public final class ToolsMultimodalBuilderExample {
    private ToolsMultimodalBuilderExample() {}

    public static Agent build(
        Role myRole,
        AgentPrincipal principal,
        LiabilitySink sink,
        AuthorityScope scope
    ) {
        return AgentBuilder.create()
            .llm(new OpenAIClient("gpt-4o"))
            .role(myRole)
            .builtInTools(
                BuiltInTool.IMAGE_GEN_TOOLS,
                BuiltInTool.TEXT_TO_SPEECH_TOOLS,
                BuiltInTool.SPEECH_TO_TEXT_TOOLS
            )
            .principal(principal)
            .liabilitySink(sink)
            .authorityScope(scope)
            .build();
    }
}
```

The agent then calls the tools from a chat turn:

```java
Agent artist = ToolsMultimodalBuilderExample.build(myRole, principal, sink, scope);
String reply = artist.chat("Draw a watercolor painting of a giraffe on Mars");
```

Every tool returns a JSON string. Each tool checks its arguments before the HTTP call, so a bad value fails with `IllegalArgumentException` naming the allowed values.

## Images: `IMAGE_GEN_TOOLS`

| Tool | Provider | Key | Returns |
|---|---|---|---|
| `dalle3_generate` | OpenAI DALL-E 3 | `OPENAI_API_KEY` | `provider`, `model` (`dall-e-3`), `urls`, `revised_prompt` when the model rewrote the prompt |
| `flux_generate` | FLUX on Replicate | `REPLICATE_API_TOKEN` | `provider`, `model` (`flux-<model>`), `urls` |
| `stability_generate` | Stability AI Stable Image | `STABILITY_API_KEY` | `provider`, `model`, `urls`, `finish_reason` when present |

| Tool | Parameters | Defaults |
|---|---|---|
| `dalle3_generate` | `size`: `1024x1024`, `1792x1024`, `1024x1792`. `quality`: `standard`, `hd`. `n`: must be `1`. | `1024x1024`, `standard`, `1` |
| `flux_generate` | `model`: `schnell`, `dev`, `pro`. `aspectRatio`, such as `1:1` or `16:9`. `outputFormat`: `webp`, `jpg`, `png`. | `schnell`, `1:1`, `webp` |
| `stability_generate` | `model`: `core`, `sd3`, `ultra`. `aspectRatio`, such as `1:1` or `16:9`. | `core`, `1:1` |

DALL-E and FLUX return provider-hosted URLs. DALL-E URLs expire about an hour after generation, so download the image if you need to keep it. Stability returns the image itself, and `urls` holds it as a `data:image/png;base64,…` URI.

## Speech: `TEXT_TO_SPEECH_TOOLS`

| Tool | Provider | Key | Models or voices | Default |
|---|---|---|---|---|
| `elevenlabs_tts` | ElevenLabs | `ELEVENLABS_API_KEY` | `eleven_multilingual_v2`, `eleven_monolingual_v1`, `eleven_turbo_v2_5`, `eleven_flash_v2_5` | `eleven_multilingual_v2`, voice `21m00Tcm4TlvDq8ikWAM` |
| `cartesia_tts` | Cartesia | `CARTESIA_API_KEY` | `sonic-2`, `sonic-english`, `sonic-multilingual` | `sonic-2`, voice `a0e99841-438c-4a64-b679-ae501e7d6091` |
| `deepgram_tts` | Deepgram Aura | `DEEPGRAM_API_KEY` | 12 voices, such as `aura-asteria-en` and `aura-orion-en` | `aura-asteria-en` |

ElevenLabs and Cartesia take a `voiceId` next to the model. Deepgram picks the voice through `model`.

All three return the same shape:

```json
{
  "provider": "elevenlabs",
  "model": "eleven_multilingual_v2",
  "audio_uri": "data:audio/mpeg;base64,SUQzBAAAAAAAJ...",
  "audio_bytes": 45920
}
```

`audio_uri` plays directly in an `<audio>` element. Input is capped at 2000 characters per call.

## Transcription: `SPEECH_TO_TEXT_TOOLS`

Each tool takes `audioPath`, the absolute path to a local file of at most 50 MB. Supported formats are MP3, WAV, M4A, OGG and FLAC.

| Tool | Provider | Key | Models | Returns |
|---|---|---|---|---|
| `deepgram_transcribe` | Deepgram | `DEEPGRAM_API_KEY` | `nova-2` (default), `nova-2-general`, `nova-2-meeting`, `nova-2-phonecall`, `nova-3`, `enhanced`, `base` | `provider`, `model`, `text`, `language`, `confidence` |
| `assemblyai_transcribe` | AssemblyAI | `ASSEMBLYAI_API_KEY` | `universal-2` | `provider`, `model`, `text`, `language`, `audio_duration` |
| `replicate_whisper` | Whisper on Replicate | `REPLICATE_API_TOKEN` | `large-v3` (default), `large-v2`, `medium`, `small` | `provider`, `model` (`whisper-<model>`), `text`, `language` when present |

`deepgram_transcribe` takes a `language` code, default `en`. `assemblyai_transcribe` takes an optional `languageCode`.

`assemblyai_transcribe` uploads the file, submits a job and polls it. Polling starts at two seconds, backs off to ten, and gives up after 300 seconds.

The OpenAI `whisper_transcribe` in `MEDIA_TOOLS` has its own 25 MB cap, and `openai_tts` caps input at 4096 characters.

## API keys

| Variable | Used by |
|---|---|
| `OPENAI_API_KEY` | `dalle3_generate`, `whisper_transcribe`, `openai_tts` |
| `REPLICATE_API_TOKEN` | `flux_generate`, `replicate_whisper` |
| `STABILITY_API_KEY` | `stability_generate` |
| `ELEVENLABS_API_KEY` | `elevenlabs_tts` |
| `CARTESIA_API_KEY` | `cartesia_tts` |
| `DEEPGRAM_API_KEY` | `deepgram_tts`, `deepgram_transcribe` |
| `ASSEMBLYAI_API_KEY` | `assemblyai_transcribe` |

A tool reads its key from the environment when it is called, not when the agent is built. You can register a toolkit without its keys. A call without the key throws `IllegalStateException`, and the message names the missing variable.

## Related

- [Catalog](catalog.md): every built-in toolkit.
- [Audio](../llm/audio.md): `WhisperClient` for transcription outside an agent.
- [Custom tools](custom.md): write your own `@Tool` methods.