Package com.tnsai.evaluation.spi

Interface Evaluator

All Known Implementing Classes:

AbstractJudgeEvaluator, AnswerRelevancyEvaluator, BiasEvaluator, ContextualPrecisionEvaluator, ContextualRecallEvaluator, ConversationCompletenessEvaluator, CostEfficiencyEvaluator, FaithfulnessEvaluator, GEvalEvaluator, HallucinationEvaluator, InstructionFollowingEvaluator, JsonCorrectnessEvaluator, KnowledgeRetentionEvaluator, PIILeakageEvaluator, PlanAdherenceEvaluator, ResponseQualityEvaluator, StepEfficiencyEvaluator, TaskCompletionEvaluator, ToolCorrectnessEvaluator, ToolSelectionEvaluator, ToxicityEvaluator, TurnRelevancyEvaluator

public interface Evaluator

SPI interface for all evaluators. Implementations score agent behavior on a specific dimension (quality, tool selection, cost, etc.).

Each evaluator is identified by a unique name and produces an EvaluationResult with a normalized score in [0.0, 1.0].

Since:

0.1.0

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static final record	Evaluator.EvaluationInput	Input context provided to evaluators.

Method Summary

Modifier and Type	Method	Description
EvaluationResult	evaluate(Evaluator.EvaluationInput context)	Evaluates agent behavior for a single interaction.
String	name()	Unique name for this evaluator.

Method Details

name

String name()

Unique name for this evaluator.

evaluate

EvaluationResult evaluate(Evaluator.EvaluationInput context)

Evaluates agent behavior for a single interaction.

Parameters:

context - evaluation context containing input, output, tool calls, etc.

Returns:

evaluation result