Package com.tnsai.evaluation.junit

Annotation Interface EvalTest

@Documented @Target(METHOD) @Retention(RUNTIME) @TestTemplate @ExtendWith(EvalExtension.class) public @interface EvalTest
Marks a method as an evaluation test. Each TestCase in the dataset becomes a separate JUnit invocation.

Usage


 @EvalTest(dataset = "rag-golden.json")
 void ragQuality(TestCase testCase, EvalAssert eval) {
     var output = agent.run(testCase.input());
     eval.assertEval(output, new FaithfulnessEvaluator(judge))
         .withThreshold(0.7)
         .passes();
 }
Since:
0.3.0
See Also:

  • Required Element Summary

    Required Elements
    Modifier and Type
    Required Element
    Description
    String
    dataset
    Path to the dataset JSON file.

  • Optional Element Summary

    Optional Elements
    Modifier and Type
    Optional Element
    Description
    String
    tag
    Filter test cases by tag.
    double
    threshold
    Default score threshold for all evaluators (0.0-1.0).

  • Element Details

    • dataset

      String dataset
      Path to the dataset JSON file. Resolved from classpath first, then filesystem.
      Returns:
      dataset path

    • tag

      String tag
      Filter test cases by tag. Empty means all cases.
      Returns:
      tag filter
      Default:
      ""

    • threshold

      double threshold
      Default score threshold for all evaluators (0.0-1.0). Can be overridden per-assertion via
      invalid reference 
      EvalAssert#withThreshold(double)
      .
      Returns:
      default threshold (default: 0.7)
      Default:
      0.7