Package com.tnsai.evaluation.evaluators

Class AbstractJudgeEvaluator

java.lang.Object
com.tnsai.evaluation.evaluators.AbstractJudgeEvaluator
All Implemented Interfaces:
Evaluator
Direct Known Subclasses:
AnswerRelevancyEvaluator, BiasEvaluator, ContextualRecallEvaluator, ConversationCompletenessEvaluator, FaithfulnessEvaluator, HallucinationEvaluator, KnowledgeRetentionEvaluator, PlanAdherenceEvaluator, TaskCompletionEvaluator, ToolCorrectnessEvaluator, ToxicityEvaluator, TurnRelevancyEvaluator
public abstract class AbstractJudgeEvaluator extends Object implements Evaluator
Base for single-call LLM-as-judge evaluators: build one prompt, send it to the LLMJudge once, parse the verdict.

Captures the skeleton that every such evaluator repeated — the judge field and its non-null contract, the name(), and the "LLM judge error: " + e.getMessage() failure when the judge throws — so a concrete evaluator supplies only its prompt, its verdict parsing, and (optionally) its input preconditions.

The judge is invoked exactly once. Multi-step judges (e.g. G-Eval's chain-of-thought → score, which calls the judge twice) do not fit this template and implement Evaluator directly.

Since:
0.13.0

  • Method Details

    • name

      public final String name()
      Description copied from interface: Evaluator
      Unique name for this evaluator.
      Specified by:
      name in interface Evaluator

    • evaluate

      public final EvaluationResult evaluate(Evaluator.EvaluationInput input)
      Description copied from interface: Evaluator
      Evaluates agent behavior for a single interaction.
      Specified by:
      evaluate in interface Evaluator
      Parameters:
      input - evaluation context containing input, output, tool calls, etc.
      Returns:
      evaluation result