Package com.tnsai.evaluation.junit

Class EvalAssert

java.lang.Object

com.tnsai.evaluation.junit.EvalAssert

public class EvalAssert
extends Object

Fluent assertion API for evaluation tests.

Usage

 eval.assertEval(output, new FaithfulnessEvaluator(judge))
     .withThreshold(0.8)
     .passes();

 eval.assertEval(output,
         new FaithfulnessEvaluator(judge),
         new RelevancyEvaluator(judge))
     .withThreshold(0.7)
     .passes();
 

Since:

0.3.0

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	EvalAssert.EvalAssertion	Assertion chain for evaluation results.

Method Summary

Modifier and Type	Method	Description
EvalAssert.EvalAssertion	assertEval(String agentOutput, Evaluator... evaluators)	Runs evaluators against the agent output and returns an assertion chain.
EvalAssert.EvalAssertion	assertEval(String agentOutput, Map<String,Object> metadata, Evaluator... evaluators)	Runs evaluators with additional metadata (tool sequences, latency, etc.).

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

assertEval

public EvalAssert.EvalAssertion assertEval(String agentOutput,
 Evaluator... evaluators)

Runs evaluators against the agent output and returns an assertion chain.

Parameters:

agentOutput - the agent's response

evaluators - one or more evaluators to run

Returns:

assertion chain

assertEval

public EvalAssert.EvalAssertion assertEval(String agentOutput,
 Map<String,Object> metadata,
 Evaluator... evaluators)

Runs evaluators with additional metadata (tool sequences, latency, etc.).

Parameters:

agentOutput - the agent's response

metadata - additional input metadata

evaluators - evaluators to run

Returns:

assertion chain