# Run evaluations as JUnit tests

`@EvalTest` turns a dataset into JUnit 5 tests: one test per case, each named after the case's id. It is part of `tnsai-evaluation`, which brings `junit-jupiter-api` with it; your build still needs the JUnit engine to run the tests.

## Write an eval test

<!-- java-contract: src/main/java/com/example/tnsai/docs/EvaluationJunitExample.java -->
```java
static class SupportEvalTest {

    static Agent agent;                                   // build it in a @BeforeAll method

    @EvalTest(dataset = "evals/support.json", tag = "billing", threshold = 0.6)
    void answersBillingQuestions(TestCase testCase, EvalAssert eval) {
        String answer = agent.chat(testCase.input(), false);
        eval.assertEval(answer, new ResponseQualityEvaluator(), new PIILeakageEvaluator())
            .passes();
    }
}
```

| Attribute | Default | Meaning |
|-----------|---------|---------|
| `dataset` | required | A [dataset file](index.md#write-a-dataset), looked up on the classpath first and then as a file path |
| `tag` | all cases | Runs only the cases with this tag |
| `threshold` | 0.7 | The lowest passing score for every evaluator |

The test method can take a `TestCase`, an `EvalAssert`, or both. The test method fails with an `ExtensionConfigurationException` when the dataset cannot be loaded or no case has the tag.

## Assert on the scores

`eval.assertEval(answer, evaluators...)` runs each evaluator on the answer and returns an assertion:

| Method | Checks |
|--------|--------|
| `passes()` | Every score is at least the threshold. The error lists each evaluator below it, with its score and details |
| `fails()` | Every score is below the threshold |
| `withThreshold(value)` | Replaces the threshold for this assertion |
| `results()` | Returns the `EvaluationResult` list for your own checks |

The evaluation input holds the case's `input` as the user input, its `expectedOutput`, and the answer you pass. It does not carry the case's instructions, tools or metadata, so evaluators that need them score as if those were missing. `assertEval(answer, metadata, evaluators...)` adds metadata, and that metadata stays on the same `EvalAssert` for later calls in the method. For full control, build an `EvaluationInput` yourself, as in [Call an evaluator directly](evaluators.md#call-an-evaluator-directly).

Every evaluator scores higher for better answers, including `bias` and `toxicity`. To check that an answer is safe, use `passes()` with those evaluators, not `fails()`.

## Keep live tests out of the default build

An eval test calls the agent, and judge evaluators call a model. Run these tests in a separate Maven profile or tag them with JUnit's `@Tag`, so the regular build does not depend on a model being reachable.

## Related

- [Evaluation](index.md)
- [Evaluators](evaluators.md)
- [Evaluate traces and live runs](trace-eval.md)