# Choose files from a folder

Pick which files in a knowledge folder an agent reads, with `include` and `exclude` patterns.

A `@Knowledge` source with a folder `path` reads every supported file under it. The `include` and `exclude` lists narrow that set. They run before the format check, so a file they drop is never opened.

## Before you start

- TnsAI 0.18.7 with `tnsai-rag` on the classpath.
- A role or agent with a `@Knowledge` source ([Declarative knowledge](index.md#declarative-annotations)).
- PDF and Office files need the optional PDFBox or POI dependency ([File formats](file-formats.md)).

## How a pattern matches

Each pattern is matched against the file's path **relative to `path`**, with `/` between folders. For `path = "notes"`, the file `notes/2026/review.pdf` is matched as `2026/review.pdf`.

| Token | Matches |
|---|---|
| `*` | Any characters inside one folder name or file name. It never crosses a `/`. |
| `?` | Exactly one character, inside one name. |
| `**` | Zero or more whole folders. It must be followed by `/` or end the pattern, so `**.md` is refused. Write it as a whole folder name, as in `**/*.md` or `guides/**`. |
| `{a,b}` | Any one of the comma-separated alternatives, since 0.18.7. `**/*.{md,pdf}` selects either extension, and `{guides,api}/**` either folder. Each alternative may use `*`, `?` and `**`. |

Any other character matches itself, and so does a comma outside braces. A pattern is anchored at both ends, so `drafts/**` means a `drafts` folder directly under `path`, not one deeper down.

The order is fixed:

1. If `include` is empty, every file is a candidate. Otherwise a file must match at least one include pattern.
2. A file that matches any exclude pattern is dropped. Exclude always wins.
3. The default skips and the ignore files act as extra exclude patterns ([What is always skipped](#what-is-always-skipped)).

### Letter case

Since 0.18.7, matching ignores letter case. `**/*.pdf` selects `B.PDF` and `c.Pdf`, and `drafts/**` also drops `Drafts/x.md`.

The pattern and the path are lower-cased one character at a time, the same way in every locale. So `I`, `i` and `İ` match one another, but the Turkish dotless `ı` matches only itself. The Greek final `ς` matches `σ` and `Σ`.

Ignore files are the exception: their lines are still case-sensitive ([What is always skipped](#what-is-always-skipped)).

### Patterns that are refused

A refused pattern fails at binding, and the error names the source and the pattern.

- `[`, `]` and `\` are not part of the pattern syntax.
- Brace groups cannot nest, as in `{a,{b,c}}`.
- A group cannot be empty (`{}`) or hold an empty alternative (`{md,}`).
- Every `{` needs a matching `}`, and every `}` a matching `{`.
- Inside a group, an alternative may end in `**` only if the group ends the pattern or is followed by `/`. So `{docs,guides/**}` works, but `{docs/**}.md` is refused.

## Examples

### Select by extension

<!-- java-contract: src/main/java/com/example/tnsai/docs/FileSelectionByExtensionRole.java -->
```java
@RoleSpec(knowledge = @Knowledge(
    name = "notes",
    path = "notes",
    include = {"**/*.{md,pdf}"},
    search = @Search(queryParam = "question")))
```

Selects `todo.md`, `2026/review.pdf` and `README.MD`, because case does not matter. Skips `plan.txt`.

### Select whole folders

<!-- java-contract: src/main/java/com/example/tnsai/docs/FileSelectionByFolderRole.java -->
```java
@RoleSpec(knowledge = @Knowledge(
    name = "handbook",
    path = "handbook",
    include = {"guides/**", "api/**"},
    search = @Search(queryParam = "question")))
```

Selects everything under `guides/` and `api/`, at any depth: `guides/setup.md`, `guides/deploy/linux.md` and `api/endpoints.txt`. Skips `blog/launch.md` and the top-level `README.md`. The single pattern `{guides,api}/**` selects the same files.

### Leave folders out

<!-- java-contract: src/main/java/com/example/tnsai/docs/FileSelectionExcludeRole.java -->
```java
@RoleSpec(knowledge = @Knowledge(
    name = "wiki",
    path = "wiki",
    exclude = {"archive/**", "drafts/**"},
    search = @Search(queryParam = "question")))
```

Reads every file except those under the top-level `archive/` and `drafts/` folders. `team/drafts/notes.md` is still read, because `drafts/**` is anchored at `path`. Use `**/drafts/**` to drop a `drafts` folder at any depth.

### Include, then exclude

<!-- java-contract: src/main/java/com/example/tnsai/docs/FileSelectionMixedRole.java -->
```java
@RoleSpec(knowledge = @Knowledge(
    name = "library",
    path = "library",
    include = {"guides/**/*.md", "reports/**/*.pdf"},
    exclude = {"reports/old/**"},
    search = @Search(queryParam = "question")))
```

Selects Markdown under `guides/`, including `guides/start.md` directly in it, because `**/` also matches zero folders. Selects PDFs under `reports/`, except those in `reports/old/`. So it reads `guides/start.md`, `guides/admin/users.md` and `reports/q3.pdf`. It skips `guides/cheatsheet.pdf`, `reports/summary.md` and `reports/old/q1.pdf`.

### Several sources on one agent

<!-- java-contract: src/main/java/com/example/tnsai/docs/FileSelectionAgent.java -->
```java
@AgentSpec(
    description = "Support agent that reads guides and tickets",
    knowledge = {
        @Knowledge(
            name = "guides",
            path = "docs/guides",
            include = {"**/*.md"}),
        @Knowledge(
            name = "tickets",
            path = "support/tickets",
            include = {"**/*.txt"},
            exclude = {"private/**"})
    })
```

Each source has its own `path` and its own patterns, and a pattern is relative to its own source. `guides` reads `install.md` but not `images/logo.txt`. `tickets` reads `1001.txt`. It skips `private/1002.txt` and `1003.md`.

## What is always skipped

Some folders and files are skipped before your patterns run. An `include` pattern cannot bring them back.

- **Folders**, at any depth, whatever their letter case: `.aws`, `.git`, `.gnupg`, `.gradle`, `.hg`, `.idea`, `.kube`, `.next`, `.ssh`, `.svn`, `.vscode`, `__pycache__`, `build`, `coverage`, `dist`, `node_modules`, `out`, `target`, `vendor`.
- **Files**, at any depth, with the exact name: `.env`, `.env.*`, `.git-credentials`, `.netrc`, `.npmrc`, `.pypirc`, `id_dsa`, `id_ecdsa`, `id_ed25519`, `id_rsa`. A file named `.ENV` is not on this list and is read.
- **Ignore files:** every `.gitignore` and `.tnsignore` in the folder adds its rules as exclude patterns. The ignore files themselves are never read. Their lines use only `*`, `**` and `?`, so a brace there is refused. They also stay case-sensitive: `secret.md` in `.tnsignore` does not drop `SECRET.md`. [Ignore files and default skips](pipeline.md#ignore-files-and-default-skips) has the pattern rules and the size caps.

> **Warning:** a real content folder named `build`, `out`, `vendor` or `dist` is skipped without an error. That includes the source folder itself. `path = "docs/build"` loads nothing, and `path = "site"` skips `site/build/` and `site/guides/out/`. Rename such a folder, or point `path` at a folder with another name.

## Check that it works

To see what a source selects, ingest it once with `new FileIngestionService().ingest(KnowledgeSourceConfig.from(knowledge))`. Each returned chunk names its file in `document().origin()`.

Every example on this page is checked that way against real folders, PDFs included. The test is `FileSelectionContractTest` in the Docs repository's `tests/java-contracts`. Run it from that folder:

```bash
mvn -q test -Dtest=FileSelectionContractTest
```

## See also

- [File formats](file-formats.md): which files a format accepts once your patterns pass them.
- [Server RAG pipeline](pipeline.md#ignore-files-and-default-skips): ignore-file rules, size caps and walk limits.
- [Declarative knowledge](index.md#declarative-annotations): declaring sources on roles and agents.